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PREFACE 

William W. Turnbull 



Both professional and public attention is focused these 
days on the concept of educational accountability. The word 
"accountability" has as many meanings as people caie lo give it and 
is often used in connection with such activities as assessment, 
evaluation, auditing, and performance contracting. 

So great is the interest -- and so meager the clarifica- 
tion of the many issues and problems involved — that the need for 
a comprehensive look at the concept prompted Educational Testing 
Service to sponsor these conferences. 

We were indeed fortunate that seme of the ir.ost knowl- 
edgeable and thoughtful people concerned with the philosophy, 
strategies, and pitfalls of accountability in education accepted 
our invitation to participate in the conferences. Each of them 
has provided a substantive and challenging contribution to better 
ueder s tanding of vhat is involved in developing and implementing 
accountability programs of integrity and merit. 

W 2 are also indebted to John H. Fischer, the conference 
ebaiman, whose contributions to education's "accountability" in 
the broadest sense have few parallels. 

Because of the urgent need for dissemination of informa- 
tion about accountabi J ity tne speakers* papers, in their pre- 
conference form, have been assembled in this booklet for immediate 
distribution, 
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INTRODUCTION 



John H. Fischer 



We had planned to include in this publication a brief 
introduction by Dr. Fischer in which he would present his views 
on educational accountability. Unfortunately, he became ill, and 
as a consequence will be unable either to prepare his remarks or 
to participate in the conferences. 
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T H K A E A N S A N D E N D S OF 
ACCOUNTABILITY 



Terrel 11. Bell 



T he Broad Concept: of Acc ountabil ity 

The whole idea of accountability is related to the propo- 
sition that v if the student has not learned the school has not taught. 
The school, cr the teacher, or someone must account for learning 
failure. But the concept of accountability goes beyond this. It is 
dedicated to the central belief that we can develop a much more 
effective education system by studying learning inputs and outcomes. 
By studying the means and ends of learning, wc may be able to, at 
long last, utilize the scientific method in teaching and learning. 

The current emphasis on accountability reflects our frustration 
about our schools. Much more is being demanded than ve have been 
able to produce. Until we solve the problem of accountability -- 
quantifying inputs and output -- little hope is seen for arriving 
at solutions to many of the great Issues facing the schools. 
Accountability in education is mure than testing of students and 
evaluating teacher competence. In its full context, we must view 
accountability as concerned with all factors related to learning in- 
put and output. 

It is important to give emphasis to this because account- 
ability has been tied closely to instructional personnel. V'e think 
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of reward* and punishment and of paying more dollars to teachers 
whose stiden s j rforr well and less dollars to those teachers 1 hose 
studetics fail to me t i standard oi expectation. Accountability 
should be apj. * ied to the materials and media of instruction. It must 
relate to time and scheduling. If should also address itself to 
questions such as: 

o What is the educational worth of a specific 150 frame 
sequence of programmed learning material in remedi- 
ating a particular diagnosed learning deficiency? 

o What is the educational worth of a video tape or a 
film? 



Accountability is also related to school administrative decision 
making : 

o What cost-benefit value can we attach to $50,000 re- 
deployed from subject matter supervisors in the central 
office of a school system to employ tutors or teacher 
aides ? 

o Do students learn chemistry in a chemistry laboratory 
and foreign languages In a language laboratory? 



Seeking reasonably objective evidence to answer these 
questions is not easy, but accountability must take such questions 
into its accounting. Seeking solutions to such problems will lead 
administrators to conclude that installing accountability systems 
might well do much to shape up the management of schools as well as 
the teaching side of the operation. Accountability locks at school 
resource deployment, materials selection, time allocating , and a 
host of other school management practices. 

Needless to say, accoun tabi li ty has many facets, forms, and 
faces. It reaches far beyond the simplistic assertion that it is 
concerned with teachers and teaching. Learners and learning reach into 
some of these management and resource deployment decisions. When the 
student fails to learn, the entire system must be introspective. 
Accountability is the word symbolic of this needed ends and means intro- 
spection . 
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Me 0. 3 urable Edu. c a K i onal G qals : 

The Key to Makin g Ace ountabil ; tv_ Opera tional 

Educational goal netting is the prerequisite for measuring 
the distance between what is and what ought to be and in monitor- 
ing progress in getting to where we are going. Our goals must be 
stated in quantifiable terms. Broad, sweeping, and idealistic 
generalities will not do if w T e seek to measure ends and those means 
that may influence ends. 

Although some goals in education will be difficult to 
quantify and respected authorities will differ on some priorities, 
there exists, it seems to me, a general consensus about many desired 
outcomes. This is represented in many almost universally accepted 
curricula found in schools across the nation. 

What is needed, then, is more precision in describing 
desired outcomes as measurable objectives that comprise much (but 
not all) of the goal structure of our schools. 

Educational goals, stated in quantifiable language, will 
provide the end we seek to attain. Surexy, this is one step toward 
objectivity in education. F.ven if we^. lack agreement on some of the 
goals of education, there are far more about which there seems to be 
1 I t t le dispute. 

Learning as an O utcome a nd Student Performanc e as_ a Pr oduct : 

Impl i c at io ns tor Schoo >1 Man ageme nt 

After our goals have been stated with precision ar.d after 
we have the quantifiable language describing our goals we step 
immediately into the input and output problem. We must, as I see it, 
conquer the frustrating problems on this battlefield if accountability 
is to mean more than a key word in a high sounding slogan. 

When does a student’s performance piove that he has attained 
a large educational goal? What factors or inputs helped — and to 
what extent did they help the student to attain the goal? How do we 
measure student performance as a product of our schools? 

As a people, we have grown up in a climate of science, where 
the scientific method is accorded the highest prestige. Since the 
beginning of this heavy emphasis* on science, its practitioners have 
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been asserting with varying degrees or confidence the maxim, "any- 
thing that exists, exists in some cmount, and consequently can be 
measured / 1 In the educational world, the controversial element of 
the basic scientific assertion — 11 and can be m easured 11 — continues 
to be a source of debate. 

I suppose we would all concede that some students fail to 
learn even in the presence of excellent teachers and some students 
learn in spite of the ineptness of some obviously incompetent 
teachers . 



To be trite, learning is difficult to quantify and measure. 

It is an even more perriexing task to identify what influences learn- 
ing and what causes its fruition. Conversely, it is difficult to 
identify causes for learning failure. 

From the student point of view, students claim — with some 
bitterness at times — that we are not consistent about accountability 
in education. They point out that teachers give exams to measure 
student learning in a given course. Letter grades are provided at the 
completion of most secondary school and college courses as a form of 
measurement. of learning . 

The grade point average determines such vital decisions as 
admission to graduate and professional schools, admission to certain 
pi‘estige institutions, and entry to better jobs with large corporations. 

i ludents see this attempt to measure and hold students 
accountable for l earning as inconsistent with claims that teaching can- 
not be measured and, therefore, teachers cannot be held accountable for 
what students learn or fail to learn. Putting it in concrete terms, 

Sf . ■“ students would ask: "If we Pi ace a 3.5 or a 1.2 grade point 
average label on a student, should we not be willing to do the same for 
the teacher?" 

The foregoing identifies issues that have been discussed lor 
a number of years. But today, educational accountability covers more 
substantive aspects of the business of teaching and learning. It is 
concerned with individually prescribed curricula and with making 
decisions on how and where to deploy scarce personnel and dollar lesources 
to gain the maximum output. What materials, methods, media and staff- 
ing patterns v.’ill result in what educational ends? r > have hard choices 
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to make and careful trade-offs to reckon with in educational adminis- 
tration. Many State Legislatures and the U.S. Congress are asking us 
for output measures, and we must account for results from dollars 
appropriated „ 

i. n the U.S. Office of Education, for example, the Secretary 
has put HEW rgencies on a management by objectives system. He holds 
monthly management review sessions. He wants, for example, a detail- 
ed tine phased action plan to show how 1.5 billion dollars in Title I 
of FSEA is going to buy some measurable progress in dioad\ entaged 
student accomplishment. The systems approach to laying out objectives 
and setting forth in dollars, personnel, and action strategy, the 
means for attaining goals is becoming a perplexing challenge in the 
U.S. Office of Education. We are, in short, being asked to regular' 
account for our stewardship. We cannot meet this challenge without 
more sophistication from the school systems of the nation in measur- 
ing student performance. Our sophisticated, scientific, production 
oriented society is demanding a more sophisticated, scientific and 
production oriented educational system. Accountability is the key 
word in all of this for it implies goal directed and performance 
oriented educational leadership. It implies analvsis of feedback and 
correction of aim to more accurately focus on our targets. 

We must, as I see it, readily concede that some of our most 
cherished educational outcomes will not be easily — if ever -- stated 
and measured in quantifiable terns. Can we, for example, quantify such 
lofty human values as enthusiasm, love, loyalty, character, and empathy? 

Schools surely want pupils to graduate from a passive state 
of mass acceptance to the more dynamic state of personal choice and 
decision. 

Such achievement, which could be most influential to i pupil's 
future, is obviously difficult to quantify. 

When wo turn ftom student performance as a product to caus- 
a;ive factors that contributed to tlu* outcome, we have even more complex 
problems in measurement. After we measure the ends, identification of 
contributory means may be possible by varying the input and observing 
the impact on the output. Consideration, however, of the complex act 
of teaching tells us that this is not easy, Teaching is a combination 
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of inter-personal mix of unique characteristics of the subject 
matter, the teacher, the learner , and the emotional climate of the 
da/. This lest element should never be discounted. A 3:00 p.m. 
Friday afternoon class in January has a different emotional tone 
to it than a 7:00 a.m. summer school class. A class of 75 In a 
lecture hall has a different setting, obviGu. \ , than an eight- 
student seminar session in a small, enclosed basement room. On 
the college campus, 3n all-male student class in mathematics for 
engineers taught by a 60-year old female mathematics professor has 
a far different climate than a mixed class in sophomore English 
literature taught by a handsome, unmarried assistant professor — 
particularly for the female students. 

None of the foregoing is Intended to offer anything new 
to this audience. ‘It does, however, point up the complexity of 
attributing means to ends in learning. 

We need to come to the task of finding out what works 
and what doesn't work after we have better mastery of measuring 
student performance as a product. To the extent that we can 
quantify our ends, educatioi will become more objective, and effi- 
cient because the manipulation of varying means will then be veri- 
fiable from the viewpoints of both educational adequacy and cost- 
benefit. This, it seems to me, is the beginning of educational 
equation making that will lead us away from so much of the guess- 
work and vitchcra . ting that still plagues school people. 

In the Office of Education, we are often asked to describe 
In detail what works and does not work in educating disadvantaged 
and culturally deprived children. We have some reports of out- 
standing Title I projects. But we still fall short in the crucial 
task of meeting the needs of these youth because we have not been 
working the ends and means equations with the sophistication 
necessary to crank cut some hard answers. 

In a Management by objectives system in education, the 
products are represented in student performance. Even if we admit 
that only some of the performance outcomes can be measured and that 
our yardstick will vary an inch or so because of other variables, 
v r e still muut accept -- as 1 see it — the proposition that such 
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quantification will give us the means to make some quite substantial 
strides toward objectivity. 

And it is the lack of objectivity that causes guesswork 
in selecting materials and media and in deciding upon teaching 
and learning strategies. It is this very lack of objectivity that 
has caused educators to assume many things that are not so. This 
runs all the way from teacher-pupil ratio decisions to how best to 
teach readir ; in the primary grades. 

V alue Implications of the Principle of Accountability 

Education must be managed by decision making processes that 
derive from objective information. In this quest for objectivity, 
we must realize that education must be humane. It must be people 
oriented. It must utilize democrat! methods characteristic of the 
free society we seek to enhance. 

The discipline and rigor of accountability must not lead 
us to a system that is authoritarian and threatening. 

If accountability is used to make teachers feel insecure, 
the application of management by objectives may result in destructive 
tensions in an already tension laden education system. We must, as 
I see it, have an open, non-doctrinaire approach that persuades much 
and coerces little. Teaching prospers most in an atmosphere of 
participatory management. 

We can surely have this and still operate under the banner 
of accountability. The strength of diversity and freedom that each 
school system or college department needs can actually be enhanced 
by less subjectivity. 

As we look at the management side of education, I believe 
that most of us recognize that accountability will bring about better 
management ot instruction. The management of instruction in most 
school systems and on most campuses is very weak and will remain 
feeble and ineffective until we can more accurately quantify inputs 
and outputs. It is my belief that this can be done without losing 
the essence of creative learning and cheating bad side effects from 
too much systems theory, if we keep our democratic values in mind 
as ve build our systems of accountability. 
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Conclusion : 

X am reminded of my favorite definition of education as I 
consider how little I really know about this topic and how much I 
have had to say about it: 1 Education is the process ot moving from 

cocksure ignorance to thoughtful uncertainty . " As we approach our 
measurable objectives, we must be sufficiently thoughtful about the 
uncertainty of what is measurable . Let us build a broad framework, 
leaving proper latitude and respect for the creative intelligence 
of both the teacher and student. There is a place for some broadly 
derived, measurable outcomes, to be established and administered by 
democratic processes. The big challenge is to build accountability 
into the system without the all too easily attained bad side effects 
of rigidity and structuring that stifles creativity and initiative. 
To this end, we should set a sensible course toward more objectivity 
without such becoming an end in and of itself. Accountability can 
be the means toward more effective learning for youth and greater 
academic freedom for teachers. But this will not be so if account- 
ability is established as the Orwellian big brother in education 
decision making. 

The value system of our society must guide our viewpoint 
and total perspective of educational accountability. We must seek 
to avoid closed systems of input and output information. School 
staff members should look upon an educational accountability system 
as a complex feedback mechanism that is reinforcing. Accountability 
systems must be non-authoritarian and non- threatening . A democratic 
society demands this level of maturity and openness. Too much has 
been said and written about accountability as an instrument to nail 
down the incompetent and fix blame for failure. If these latter 
emerge from the system, they will ue by-products of lesser signifi- 
cance than the prime purpose of bringing about vital infermation 
about decision making in the teaching and learning process. 
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ISSUES IN IMPLEMENTATION 

I 



Nolan Estes and Donald R. Wald rip 



THEIR LIVES AND OUR CAREERS: 

ACCOUNTABILITY AS A FAIR TRADE IN EDUCATION 

Future historians of education will probably be able to 
identify very precisely the origins of the drive for accountability. 
Speaking only for Dallas, I can say that it began with acute 
frustration. 

For five years — ever since the passage of the Elementary 
and Secondary Education Act gave us the extra financial boost we 
needed to develop compensatory programs — we had been trying all the 
old tricks and most of the new to improve achievement among those 
children whom we call the "culturally disadvantaged." Along with 
other school districts all over the country, we bought shiny new 
hardware and clever new software; invested in workshops and seminars 
for our teachers; sent our kids to concerts and museums and factories 
and even ■— courtesy of Braniff Airlines — up over the city in 
planes. Ia sum, we waved the banner of innovation as energetically 
as anyone. 

Naturally, even though we got a considerable boost from 
Title I and other forms of federal aid, our costs went up. They 
tripled in the last 10 years — mainly because of new construction, 
salary increase^, and improvements such as air-conditioning; but 
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partially because we asked the citizens of Dallas to stretch federal 
dollars with their own. And when we totalled the results of this 
financial exertion on the part of the taxpayers, and of the spiritual 
exertion on the part of our teachers, we found we didn't have much 
to be proud of. 

Our target had been those schools in which students were 
averaging only a half-year's achievement gain for every full 
scholastic year. By the time we finished, we had not managed to 
improve on this sad record; in fact, some of our Title I schools 
were worse off in 1970 than they had been in 1965. 

Any sane school superintendent is reluctant to hang out 
his dirty linen for public viewing. I cannot suppress a certain 
sense of embarrassment even now, as I speak. All that gives me 
courage to do so is the knowledge that virtually every other large 
city school system in the country lias had the same experience as 
Dallas, Five years and five billion dollars after Title I was 
passed, we still have not learned how to break the cycle of under- 
achievement that sees children from poor homes do poorly in school; 
find poor jobs or none; marry — and then send their own poor 
children to school. 

But though this failure remains constant, some things have 
changed in education — notably the public latitude toward those 
who run it. Ten years ago, we educators confidently asserted that 
we knew how to cure educational illness. All we needed was enough 
money to lower pupi 1-teacher ratios, put a library in every school, 
an overhead projector in every classroom, and so on and so forth. 

Our prescriptions for educational excellence were based on tradi- 
tional notions that went uncha llenged because a stingy public had 
never allowed us to try them. 

During the 1960 r s , we got a chance to try then; not as 
much of a chance as we would have lik.d, perhaps -- too many school 
systems spread Title 1 funds around so thinly that the extra money 
could not have any impact. Nevertheless, we were given a reasonable 
chance -- and the results did not justify the investment. And to- 
day, it is clear, the public does not lelieve it is getting its 
money's worth from public education. 
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Thus there is a public frustration as well as a profession- 
al frustration behind the driv^ for accountability. Public school 
sys ten's have developed extremely precise methods of accounting; most 
of them can tell you to the penny how much they spent for teachers' 
salaries, textbooks, red and blue litmus paper, and the wax on the 
gym n a s i um floor. 

But they cannot tell you what this investment pr oduced . 

Our focus in educational accounting has been on input , not output . 
Professor Dwight Allen of the University of Massachusetts has quite 
properly criticized the accounting methods of school systems as be- 
ing irrelevant for purposes of devising new educational strategy. 
Fer-pupil expenditures do not really tell us what it costs to 
ed ucate a student: all they telJ us is what it costs to Veep a 
student s eat ed for a year. 

A much more relevant measure, Dr. Allen argues, would be 
a "Teaming-unit" cost — the total sum, including teacher's salary, 
portion of total building expense, cost of textbooks and other 
learning materials required to move a student from one skill-level 
in reading, writing, or math to the next highest level. These costs, 
moreover, would vary from one school to another; they would be higher 
in a school with a majority of children from low-income, black or 
Spanish-speaking families than they would be in a school with a 
majority of white children from upper-income homes. 

Developing such a new accounting system W'ould enable 
educators to she the public how much lea rnin g was produced by a 
cczi-jin amount of investment. It vcould, moreover, enable educators 
to shift resources back and forth within a budget — testing, for 
example, the value of teacher-aides in one classroom against the 
value of educational technology in another and of programmed texts 
in a third. In each case, input would be related to ou t pu t — and 
educators who prescribed various teaching strategies w>ould be held 
accou ntabl e for the results they produced. 

Accountability is, in essence, a statement of policy. It 
states Lhat educators will accept responsibility for their perform- 
ance — or lack of it. It implies that there is a contract between 
schooj personnel and the public, and that that contract involves 
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more than showing up for work on time. It accepts the. fact that 
culturally different backgrounds make the task of educating more 
difficult, but it assents that, as professionals, educators can 
overcome — or will l_earn to overcome — cultural difference. 

Now statements of policy are fine things, if for no other 
reason than that they look nice framed on a wall. But if a state- 
ment of policy is to be a genuine progr am rather than just a fashion- 
able enthusiasm, it must be translated into a strategy — a set oi 
practical steps for turning an idea into a rea lity . 

Performance contracting is one such technique. It is not 
the only one. Voucher plans are another — and so, indeed, is any 
systematic effort to relate educational effort to student achievement. 

Our sense of frustration in Dallas led us to try perform- 
ance contracting. Our interest in it led us to two distinct pro- 
grams — one financed by the Office of Economic Opportunity, and 
the other by Title I. I wish today to describe the Title 1 program 
because we controlled it from the start: chose the student popu- 

lation, outlined the performance criteria, wrote the requcst-for- 
proposals, defined the conditions under which any successful con- 
tractor would have to work, and negotiated the final contracts. 

The entire process has been carefully monitored by the 
most precise scientific methods. According to our most recent 
figures, for example, every administrator involved lias lost an 
average of 13.1 pounds, given up 46.3 percent of his weekends, and 
antagonized 75 percent ol his wives to the point where 100 percent 
of them threatened to go home to mother an average of 3. A tines. 
Nevertheless, innovation marches on in Dallas. 

First, a note on the GEO program. It involves about 600 
students in grades one through three and sevm through nine in two 
schools; these students are matched with another 600 in a control 
group. The subject areas are reading and mathematics, both of 
which were subcontracted by OHO to QualiLy Education Development, 

Inc., of Washington, D. C. Contracts for two service components, 
audit and management support — l’JJ explain these t»rms a little 
later on — were awarded by OEO to B-tteile Memorial Institute and 
Education Turnkey Systems, Inr. 
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The two programs resemble each other in principle, of 
course; the major distinction is that OHO designed one program, 

Dallas the other — hence I feel 1 can discuss the Dallas program 
with no re authority. 

First, the target group. Last May, we ran an analysis of 
underachieving high school students and selected a group whom, on 
the basis of our experience, we believed were highly susceptible to 
dropping out. By August 31, the first day of school, our predictions 
were proven unfortunately accurate: fully 50 percent did not show 

up. We divided the survivors into an experimental group of 960 and 
a control group of 700. The experimental group were all students 
in grades nine through twelve attending f^ve Title I high schools. 

We decided the program should concentrate on three kinds 
of instruction: first, basic skills — communication and mathemat- 

ics, second, occupational skills; and third, achievement motivation 
— helping youngsters develop a determination to succeed. 

The characteristics of both experimental and control 
groups are as follows: they were 4.8 standard scores below the 

national 50 ' percentile in reading, 6.2 scores below on vocabulary, 
and 4.9 scores below on mathematical skills. Their teachers and 
counselors indicated that each seemed to lack any desire to 
succeed in school, ot any realistic goals in life. 

When we set up this new shop, then, we chose to go after 
Lhe toughest customers. Long before we picked them, however, we 
began thinking about the kind of program we would ask contractors 
to bid on. We started our planning in November 1969, with a Flan- 
niag Advisory Group that comprised 30 people — and 1 think it 
important to describe this group. 

The membership included only five employees of the school 
district: two central staff administrators, a principal, and two 

teachers. The other 25 included the president o r the Classroom 
Teachers of Dallas, which is the local NKA affiliate; seven students 
and ex-students; one school hoard member; and tlu» rest, residents 
of the target neighborhoods , representatives of local colleges, 
local businessmen, and officials in Dallas civic agencies. 
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The cynical way to view this is that we were trying to mini- 
mize opposition — and that, indeed, was one of the fringe benefits. 
Performance contracting seems by implication, at least, to impugn the 
competence of teachers, and one might expect their representatives 
to oppose it. But we are fo'tunate in Dallas to have NEA representa- 
tives who are equally alert to the interests of their members and 
to sound ideas for improving education. They agreed that performance 
contracting was a concept worth testing. Perhaps, they felt, it 
night be a step toward training teachers to aim for performance. 

Yet minimizing opposition was not our principal objective 
in expanding the membership of the Planning Advisory Group. It is 
difficult for any educator to admit that laymen might know a thing 
or two about educating, but we decided to investigate the possibility. 

Our humility paid off. Among many other benefits, it led 
us to include a somewhat offbeat course in the occupational training 
portion of our reques t-for-proposals : drafting for girls. That 

suggestion came from the businessmen on the advisory group -- and 
every girl enrolled in the drafting program has already been spoken 
for by a local industry. 

P>y February 1970 , the Planning Advisory Group had helped 
us develop a "wish-list”; what we hoped the contractors could do 
for us. By April, we had refined that l^st into the RFP. We held 
a pre-bid conference in May, and chose the successful contractors 
in July. 

Now -- what had we asked for in the RFP? 

The total list of performance criteria and conditions is 
much too exhaustive tc repeat here. The most important requirements 
— those which, I believe, you will be intmusted in — are these: 

First, in mathematics and communications, the students would 
have to gain 1.4 grade-levels in one scholastic year — in contrast 
to the 0.5 grade- levels this particular population had been gav.jing. 
Pavnont to the contractor wouid be based on individual student gains; 
unless every student achieved a 1.4-year gain, the contractor would 
not be able to recoup his costs. 

Second, in achievement motivation, the contractor would 
have to reduce dropout rates below those of U.S.O.h.’s five most 
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successful Title VIII dropout prevention projects throughout the 
United States, The retention rates, however, would not be based on 
attendance in the achievement motivation classes — since all a con-* 
tractor would have to do to maintain high attendance would be to 
’’ake these classes fun. Rather, measurement of the effectiveness 
of the achievement motivation classes would be based on attendance 
in the math and communications classes. 

Third, with regard to occupational training, we could not 
define performance criteria as strictly as we could with the other 
two components. The essential test of occupational training is 
employability — but this is affected by economic conditions as well 
as by educational excellence. However, we did the best ve could to 
specify performance standards for this component by enlisting 25 
local companies to work with the contractor; they participate not 
only in the actual training, but also in judging the quality of the 
program . 

The New Century Company, a subsidiary of the Meredith 
Corporation, won the contracts for communications and math. Thiokol 
Corporation won the contracts for achievement motivation and occupa- 
tional training. We also requested proposals for two other compon- 
ents: audit and r auagcment support. 

Audit, essentially, is intended to k urn everybody honest 
-- to prevent a repo i Lien of the unfortunate experience in Tex- 
arkana. he wanted an outside agency to approve the tests given to 
experimental and control group students throughout the program; to 
check our research design so that we could appraise the effective- 
ness cf various treatments, singly and in combinat ton ; to ascertain 
the reliability of data; and, finally, to certify the results so 
th'it the contractors could be properly compensated. 

In contrast to the instructional components, which were 
contracted for on a penalty-incentive basis, the audit contract 
was for a fixed fee, We chose Educational Testing Service to pro- 
vide the audit. 

Manager ont support, as the name implies, is to help out 
management -- in this case, the I>al las school system. Verformance 
contracting is new to our staff; alJ of then nave full-time duties. 
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and we did not want to divert them to an unfamiliar job. Hence we 
contracted with the Council of Great City Schools — again for a 
fixed fee — to provide a supplemental staff that would act as 
liaison between the school system, the contractors, and the auditor. 

In addition, the Council of Great City Schools felt that 
placing a few of its representatives on our staff temporarily would 
increase their expertise in performance contracting. In a sense, 
even though their people have significant experience in this area, 
they would be serving an internship — learning along with us so 
that they could later help other school systems. 

The last aspect of performance contracting that I feel you 
should know about is the "turnkey" aspect. The three instructional 
components of our program -- math, communications, occupational 
training — employ the contractors’ methods and materials, but they 
employ Dallas personnel. We insisted on this in our RFP . Moreover, 
we insisted that the contractors* programs be so designed that they 
could be adopted throughout the school system if wo elected to do so. 

That is what "turnkey" means. Thus performance contract- 
ing can be viewed not only as a tool for improving student achieve- 
ment, but as a tool for improving the effectiveness of teachers. 

Each contractor has agreed to train our teachers in his methods if 
those methods work. Each has also a, reed to supply us with his real 
expense figures, so that we can ippraise the cost-effectiveness of 
his program. We expect that eacli of them will make a profit; we’ve 
signed the contracts, and if they can deliver, we don’t care how 
much each of them makes. But we do want to be able to compare t heir 
learning-unit costs against ours, sc wo can decide whether their meth- 
ods can be extended to othes students within our budget restrictions. 

In connection with tho "turnkey" aspect of the program, I 
venture the opinion that performance contracting poses no threat to 
any school district’s teachers. But it does pose a threat to teacher- 
training institutions. If Thiokol or M w Century or Jim-Dandy 
Educational Systems can teach teachers to teacn potential dropouts 
to read, after all the tenured Ph.D.’s in cur universities have so 
resoundingly failed — then, I predict, we will see a lot of Ph.D.’s 
out of work duiing the next decade. 
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It is too early in our experiment to judge the results. 

We do_ know that our target population has a much higher attendance 
record than their controls; these youngsters whom we identified as 
probable dropouts — p robable , not possib le — are showing up 87 
percent of the time. 

This figure offers hope, but we are not resting too much 
on it. Like so many other promising ideas, performance contracting 
may fall flat on its face. In the meantime, however, we feel we’ve 
got hold of something that deserves a thorough, careful trial; that 
six months from now. or twelve or eighteen, we can go to the citizens 
of Dallas and say. "Here’s where X amount of your dollars went, and 
here’s the amount of difference which that investment produced. Now 7 , 
how about giving us Y amount of additional dollars so we can produce 
that difference for Z number of additional kids?” 

We feel we owe this to the citizens who are investing 
their taxes in the special knowledge which professional educators 
claim to possess. More Important, we owe this to the parents w T ho 
are investing their chi ldren in that special knowledge. 

Most important of all, we owe it to the students, for they 
are investing themselve s . Whether they know it or not, those chil- 
dren whom we term the "culturally disadvantaged" place most of their 
hopes for the future on the line when they enter our schools. 

Accountability asks educators to place their careers on 
the line. Since our students had no choice ot schools, but w 7 e had 
a choice of careers, this seems to me a fair trade. 
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ISSUES IN ; M P I EMEN ATION 

II 



Robert W. Locke 



ACCOUNTABILITY YES, 

PERFORMANCE CONTRACTING HAY BE 

To start with semantics: 

1) By accountabili ty , I mean the broad concept of 
establishing educational goals and looking at results. 

2) By performance contracting, I mean the relatively 
narrow application of accountability, in which educa- 
tion companies get paid according to the achievements 
derived from their programs and services* 

3) By project management, I refer to a less complex 
relationship in wh? ch companies provide the same 
programs and services, but for fixed fees. 

My assignment is to discuss the problems of these various 
relationships between schools and companies, but I shall also con- 
sider the great potential of accountability as an operating concept 
for education. 



Per f ormance Co ntrac ting 

Contrary to vhat you have read in the papers, I belitve 
that most education zompaints look upon perform nice contracting as 
an undes i rab 1 e way of doing business. For r-jn panics with carefully 
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researched programs and the competence to train teachers, it is not 
particularly risky because they know wnat kinds of results thay can 
achieve. However, it puts them in a straight jacket that makes 
performance contracting less desirable than the same work done under 
a simpler contract. The reason why certain companies, such as my 
own, have responded to the recent surge of RFP's is simply that they 
ha^e the programs, they can provide the services, and they are will- 
ing to take the risk in order to get the business. A customer is a 
cus tomer . 



T t is worth noting that many large and well-run companies 



have not sought to win performance contracts, either because they 
consider the risk too high or simply because they have reservations 
about their ability to perform the requisite services. 



The companies that are willing to make performance contracts 



— aid perhaps all education companies — wouLd surely agree on these 
things : 



1) That results in education cannot be guaranteed. in 

the fall and winter of Texarkara there weie some mis- 
guided claims about programs that could "produce 
grade-level independent readers and writers by the 
end of the first grade" of some such, but virtually 
all companies understand that intellectual processes 
cannot be guaranteed in the way that soapmakers 
guarantee cleanliness. 

?) They understand also the critical importance of having 
reliable data around whi^h to construct contracts. 

More on this point later, but the lack of sufficient 
data is probably the main reason why some responsible 
companies have been reluctant to make performance 
contracts . 

3) They also recognize that performance contracts make 
more sense for innovative programs than for conven- 
tional ones , It is hardly worthwhile, for either 
school districts or the companies, lo write .such 
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involved agreements merely for the purchase of text- 
books. It makes much more sense for the installation 
of complex new systems of instruction for which the 
learning environment will have to be reorganized and 
the teaching staff retrained. This may be the chief 
value of performance contracting, because innovative 
instructional programs are very difficult to install 
and yet hold mu« r . promise for the improvement of 
education . 

I suspect, in short, that the education companies have 
much the same general views of performance contracting as the school 
systems that wish to hire them. 

But what do they worry about when sitting nervously across 
the bargaining table? I can’t pretend to represent the position of 
any company except my own, but I suspect that most companies have 
much the same objectives. 

1) Let's start with money and get that unpleasant subject 
out of the way. The nature of the relationship dic- 
tates that the companies price their performance bids 
Mgher than when selling their materials, equipment, 
and systems off the shelf. Performance contracts 
require extensive — and expensive — services that 
ere normally performed by the school system itself. 
Foremost among th^m is teacher training. These 
services cost money, and they will cost more if pro- 
vided by the companies than if provided by the schools 
themselves. (The companies generally pay better, and 
they will expect to get return on their costs.) 
Remember that this is basically a services contract 
because the materials and equipment can be purchased 
at catalog prices without the contract. The profit -- 
or lack thereof -- on the performance relationship 
depends on how much the contractor spends on services 
and in turn gets paid for them. 
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2) Nevertheless, I wouldn't get too uptight about the 
prospects of paying more money, because the tneory of 
performance contracting is that educational results 
will be better. Remember that the only way a perform- 
ance contract can cost less is to fai'.. Conversely, 
it will be relatively economical if it succeeds. 

If the services provided, especially helping your 
teachers to use a new program effectively, are per- 
formed properly — and if we jointly succeed in mov- 
ing the achievement curve in the right direction — 
the investment will produce a good return. 

3) Companies will differ, incidentally, in how they 
construct their prices in performance contracts. The 
simplest model is to double the price of the materials 
rnd then accept no payment for any student who falls 
below a given objective — say grade-level reading. 

The most sophisticated is to price the materials and 
services separately: the materials at their catalog 

prices and the services according to a matrix of learn- 
ing objectives and achievement levels. The first 
strikes me as basically irresponsible because it 
smacks too much ef the money-back guarantees offered 
for simpler products like soap -- and I would hate to 
see the education business sink to that level. The 
second is much more appropriate because it relates 
directly to the objectives of the program, the 
services that will be provided, and the performance 

of the s t udents . 

Besides money, what do t He companies look for? 

1) One has to do with the objectives of the program. 

The more carefully they have been developed and the 
more clearly thov can be stated, the better. Hie 
companies do not want to develop obj e ct ives for the 
schools . Nor should Jiey. Instead they would like 
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to do business with school systems that can determine 
performance objectives and state them in understand- 
able terms. 

2) Likewise, they want to cJo business with school dis- 
tricts that have a sophisticated understanding of 
evaluation. I hope it is becoming clear to all that 
the progress of individual students towards specified 
learning objectives cannot be effectively measured 

by tests that are normed to group performance. While 
recognizing the value of standardized achievement 
tests we must recognize also the crucial importance 
of developing criterion-referenced tests for the 
evaluation of individual progress. The lack of such 
tests poses a problem for performance contracts now 
underway or being negotiated. Until criterion- 
referenced tests are generally available we shall 
simply have to do the best we can with less appropri- 
ate measures of performance. 

3) In quite a different area, we are interested in what 
part the school's regular teaching staff will have 

in the project. The more the better. Quite frankly, 

I doubt if many companies are interested in proving 
that they can do a better job of teaching your chil- 
dren to read than your own teachers can. And 
obviously it would accomplish very little for educa- 
tion in general if that were the only outcome of per- 
formance contracting. 

Instead, we would like to help your teachers do 
a more effective job of teaching reading, or whatever, 
using our materials and learning systems. 

For Mr. Shanker’s benefit, we have no intention 
of participating in an effort to by-pass union con- 
tracts; nor do 1 see any way in which that can possibly 
benefit the Schools. 
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A) We are also interested in the school's plans to work 
with the community, and especial' 1 ; the parents of the 
children in the program. This is especially important 
in the black community, as in both Washington and Los 
Angeles; and in the brown community as in Los Angeles. 

It is an activity in which the companies can help and 
perhaps a good way of justifying their PR departments. 

5) Finally, we are greatly concerned a’ out the length of 
the commitment. Fundamental changes in the process 
of teaching and learning are not likely to be made 
quickly, ana short-term performance contracts are not 
the way to bring them about. This is a serious con- 
cern because many of the projects have been short- 
term and there has been created the false expectation 
that ai. education company can set up, operate, and 
leave in good working order a new instructional system, 
all within the space of a year. Such a program can 
undoubtedly produce good results within that year, 
but it stands to reason that a longer commitment is 
needed and that the company should not be absolved 
of its responsibilities at the end of a year. 

These, it seems to me, are the main concerns that any 
contractor will have, and I hope they respond intelligently to the 
needs of the schools. 

Project Management 

Let me point out that schools can buy the same programs and 
services from the education companies without the complicat lens of 
performance contracts. In fact, they can specify exactly the same 
objectives and ask for the same services, but negotiate contracts for 
a system of fixed fees. Th ' * means giving up the sliding scale feature 
that may have some potitical sex appeal, but it . much simpler and 
should produce equally good results at the same or less cost. 




30 



Texarkana 



Before turning to some broader concerns about accountability, 
and at the risk of stepping on some toes, I should perhaps say some- 
thing about Texarkana. 

1) It was naive to award that contract to a company that 
did not have a complete and well-tested program. 

2) The blame for that error in judgment must be shared, 

I suspect, by the two school districts, by the Office 
of Education, and by the consulting firm that helped 
to construct the program. 

3) The monitoring arrangements were obvioucly inadequate. 

It is ironic tnat it took a student to discover that 
test items had been written into the program. 

4) We should be clear about the difference between teach- 
ing to the tert and literally teaching the test. The 
former is done, at least subconsciously, by most 
teachers, and a case can be made that it is education- 
ally sound. Writing test items into the program, on 
the other hand, is probably not educationally sound. 

It was certainly not ethical. And just incidentally 
it was probably a case of copyright violation. 

5) But the greatest shame of Texarkana was that the first 
and mos c visible experiment in performance contract- 
ing was so seriously flawed. 

I might add that a division of McGraw-Hill, Educational 
Developmental Laboratories, has been awarded the second year of the 
Texarkana program and now has the challenge of doing the job proper- 
ly. We expect to do so. 

Accountability 

It must be obvious by now that my view of performance con- 
tracting is somewhat ambivalent. However, I have no ambivalence 
about the concept of educational accountability. I believe that wc 



simply must pursue the concept for the potential good that underlies 
it. 

1) It strongly supports educational innovation, and in 
a sound, practical way. 

2) It requires a foccs on the goals of education, and on 
the matching goals of instructional materials and 
systems. 

3) Perhaps most important, it puts the emphasis on the 
processes of teaching and learning, by considering 
what individual children already know, what they need 
to learn, how best they can learn, and how their prog- 
ress can be measured. 

But there are some large issues to resolve, and we are a 
long way from having the answers to all of the thoughtful questions 
that are being rai! 3d about accountability in education. 

Will I ndustry Help ? 

As those questions get raised, can the education business 
be counted upon to help answer them in che public interest? 

Representative Edith Green has expressed concern about the 
dangers of an education-industry complex, and I think her concern 
is legitimate. There are potential dangers in the complex relation- 
ships that are developing between schools, federal and state govern- 
ments, and corporations of all sizes. Furthermore, since the educa- 
tion business is only the private sector of a publi c enterp rise , it 
would be irresponsible for the public not to be concerned. 

But let’s make sure that the relationships develop in 
such a fashion that the public interest is served. 

What we need most of all is a s>‘t of standards for the 
work dore by industry -- and also by the not- tui-prof it educational 
organizations like ETS . It should be neither a fatuous code of 
ethics such as proposed some years ago by the project ARISTOTLE 
people, nor an overly precise set of speci f ications like the school 
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building codes that in many states have limited innovation in school 
architecture . 

Rather, it should be a well-reasoned set of minimum stan- 
dards for the ways in which things should be done: 

1) The extent to which instructional materials should be 
field-tested, for instance. 

2) What Hinds of technical data should be provided when 
new programs are put on the market. 

3) To what extent the supplier should monitor the 
installation of his program. 

4) And how programs should be evaluated. 

In other words, how the pro cess should be carried out. 

I do not believe that proper standards of this sort would 
inhibit the work of the companies or the development of their 
relations with the schools. Instead they would codify what both the 
companies and the schools already know should be done, and what the 
best of them ate attempting to do. 

And it nay in the end help to make accountability a 
fundamentally important development in education, and not just the 
latest in a series of panaceas. 
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POSSIBLE EFFECTS 
ON INSTRUCTIONAL PROGRAMS 



Albert Shanker 



I think the first thing that needs to be said about acccu 
ability from the point of view of the teacher is that the concept i 
very much feared. It is leared because accountability in its recc;'i | 
thrust to prominence has had at least three separate meanings. 

The first meaning is associated with the schools where t 
parents say, "You. the teachers, are paid to teach. Our children 
have been going to school year after year and they are falling fui 
and furilnr behind. We demand that you be accountable to us. If ; 
children don't h ain we demand the right to remove you.” So, in t 
first sense, c cccunf abi 1 i t y views the teacher as a hired hand, or u 
hired mind — or both — of a group of parents. Thus, accountabil 
essentially means the right of that group to pick and choose, to i 
tain or get rid of those whom it wants to; whether on the basis oi 
adequate or inadequate information, knowledge, or judgment. 

The second meaning derives from the great desire to l nti 
educational expenditures. How is the school accounting for t lie 
dollars that we are spending for education? How do we. know we aiv 
getting our money's worth? 
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The third meaning of account abili t / deals with the develop- 
ment of professional standards. For example, there is a oody of 
agreement in other fields, such as medicine and law, as to what 
constitutes competence and incompetence. 

The fears of teachers, then, are dependent upon which of 
these three meanings is used in a given accountability effort, and 
the manner in which the objective associated with that meaning is 
ach ieved . 

Teachers are also deeply concerned about the concept of 
innovation, which is so frequently associated with accountability. 

They have learned through years of experience — and rather bitter 
experience — that educational innovation in the American public 
schools has nothing to do with the improvement of education. 

It is, instead, a kind of public relations device whereby 
the reigning political power — whether it’s a school board, or the 
principal or school superintendent trying to convince the community 
that he or she is a bright, shiny individual doing ail sorts of new 
and creative things — brings out all kinds of ideas T .<hich force 
teachers and children and others to march in different directions. 

A year later, that lot are dropped as a new set of innovations are 
produced like rabbit' from a hat. These innovations, rather than 
being honest attempts at educational improvement, are really public 
relations efforts. 

Further, there is a great discrepancy between, on the one 
hand, the educational change and innovation expected by the educational 
establishment and the New Left critics, and, on the other hand, what 
is actually expected from teachers in the classroom. Namely, that the 
teachers are expected to maintain a rather high degree of order in a 
rather unusual situation. That is, you place 30 youngsters in their 
seats at 8:30 a.m., and the teacher’s prime responsibility is to keep 
them relatively quiet, relatively immobile for a long period of time. 

Research has shown that tnis expected degree of order is 
based on a series of sanctions which the teacher has developed. And 
the students, in turn, have developed understandings with the teacher. 
They know, for example, that if they are not too disruptive, if the? 
whisper quietly, the teacher will agree to ignore them, to withhold 



the sanction. Such a relationship can only be maintained ir there is 
a relative amount of stability and continuity in what goes on in the 
classroom. 

Unfortunately, change and innovation upset these understand- 
ings, with an ensuing risk of chaos and disruption in the school. We 
must remember that when an observer — be he parent, principal, or 
school board member — walks through the school, he rarely notices the 
wonderivl innovations. But he’s sure to notice how many kidr are yell- 
ing and running around! It will not then be a satisfactory answer to 
say, ,T I was trying to innovate today, but it didn’t work out. The kids 
didn’t quite understand.” 

So, the teacher risks something with innovation. He risks 
those very understandings and relationships which tend to maintain the 
orderliness and quietness that parents seem to want. 

Teachers are also disturbed by the frequent association of 
accountability with something called "teacher mot iva tiom, ” a doctrine 
which holds that many teachers fail to reach the children because 
they don’t really want to. These teachers are accused of just being 
job holders — not really trying and not really wanting to do anything 
productive Hence the calls for an individual system of punishments 
and rewards, geared to the children’s progress. 

This view of accountability poses a great threat, because, 
to be honest, most teachers aren 1 t doing the best they can. And for 
a very simple reason: they don’t know any other way of doing things. 

They are the victims, if you like, of a system that has seen eight 
thousand new teachers move into New York, for example, every year for 
the past twenty years. These new teachers, drawn from many different 
colleges and universities, are a remarkably diverse group: Catholics 

and Protestants, Jews and nonbelievers, blacks and whites, liberals 
and conservatives* Yet, after four weeks of teaching in New York City 
it is almost impossible to distinguish the newcomer:- from those they 
replaced. Which leads to a rather obvious conclusion: With the 

exception of the few outstanding figures who somehow operate cn an 
individual basis, the overwhelming majority of teachers do whit the 
school as a system compels them to Jo. 



lu these circumstances, it obviously makes little sense to 
talk in terms of individual rewards and punishments. So it is a threat 
to say you are going tc apply individual rewards and punishments when 
the individual has no freedom to change his ways. It is exactly for 
this reason that writers like Holt, Goodman, and others are rejected 
by teachers. They are rejected because of the arrogance of the writing. 

Essentially, these New Left critics are behaving like a star of the 

Metropolitan Opera who criticizes his audience for being unable to sing 
as well as he does. Many of these books are written by self-proclaimed 
star performers for no other purpose than to say, "Look at all those 
lowly characters out there who are not as artistic as I am!" That, of 
course, is not very helpful to the ordinary practitioner. 

Another difficulty with accountability lies in our present 
failure to use . ’ch knowledge as we already possess in a few vital 
areas. 1 will cite just two examples. The first concerns the findings 
of Benjamin Bloom, and others, that a major part of intellectual 

development urs between the ages of two and five. Despite almost 

universal agreement on this point, there is practically no movement 
on the part oi government — federal, state, or local — to develop 
in education program at that level. The second example concerns 
juni u high schools. We've had junior high schools for about fifty 
years, yet it is tragic to reflect that, even today, ninety-nine per- 
cent of the students who enter junior high school without knowing how 
to read, ’ri^e, or count, leave in the same plight. School, for one 
of these youngsters, represents a context of failure, and in 
consequence, he does one of two things: He either drops out internally 

by just sitting in the back of tne room, and will leave you alone if 
you leave nin alone; or, he lashes out and becomes the viulent and 
disrupt i >•" youi.gster that we see every day. This we know only too 
well, but over all these years nothing has been done to create an 
alternative- model of education for such youngsters to identify with. 

We know, l r we do not act. 

Witu all these problems arrayed against it, how do^s one 
get teachers ' i accept this odd notion of accountability? To begin 
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with* the first two conceptions of accountability that I mentioned 
must be firmly opposed. I tnlnk it is quite clear that teachers are 
going to reject the notion that they are just hired hands. Secondly, 
they are not overly concerned with arguments about budgets. Teachers 
will react negatively to statements that they must change their ways 
either because few or many dollars are being spent. 

The third concept of accountability as being the development, 
with other groups, of common objectives is, I believe, acceptable to 
teachers, because strictly speaking it is not for teachers alone to 
determine what the objectives of education are. Nor are teachers as 
intractable on the subject as might be supposed, for they have already 
roved in this direction. In June 1969, the United Federation of 
Teachers in New York City became, I believe, the first organization 
in the country with a contract clause stating that the Federation and 
the Board of Education would work together to develop objective 
standards of professional accountability, in cooperation with parent 
groups, community boards, universities, and other interested parties. 
There have been a number of meetings to this end, and, believe it 
or not, these groups which had been on opposite sides of the barri- 
cade in 1968 — and which are still not friendly to each other — these 
same groups reached unanimous agreement on whet they wanted. 

The proposal has two parts. The first follows a nana^ement- 
by-obj ectives approach, with teachers, parents, students, community 
boards, the Board of Education, and supervisors at all levels develop' 
ing agreed-upon objectives. Objectives which are not so narrow as to 
turn children into machines, but also not so broad as to make measure- 
ment impossible. 

The second part of the program is perhaps the largest research 
design ever put together. Its aim will be to identify the districts 
within the city, the schools, the programs, the materials — the 
individual, even -- that are doing something to reach the objectives. 
And, more important perhaps, it will also try to identify the factors 
which have nothing to do with the objectives, which are neutral; and 
those which aie dysfunctional. This part of the program will include 
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social, family, economic, and educational information in a form unlike 
anything seen hitherto. 

The ambitious, far-reaching nature of this proposal suggests 
an important principle that is, perhaps, not too well understood as yet. 
But we must all come to understand it, eventually, if we are to make 
any progress with accountability. Simply stated, the principle is 
this: Where accountability is concerned, no man is an island. 

Teachers do not work in a vacuum, a controlled environment 
with all random factors controlled. So it is impossible to develop 
a design that will tell you what the teacher should be doing, or 
which practices are good and which bad, without considering those 
random factors, or outside influences, that limit the performance of 
even the best of teachers. The individual student, his family, his 
socioeconomic background, and the school system itself, must all be 
held accountable in degrees yet to be determined for everyone involved. 

When this principle is clearly understood and freely accepted 
it will be easier for teachers to believe that a system of professional 
accountabili ty does not, necessarily, imply an individual threat. 

For the inevitable effect of such a system will be changes in the 
structure of the school and of the school system in which it operates. 
Changes that will break the vicious circle in wh? ch each year, for 
twenty years, those eight thousand new teachers have found themselves. 
Changes that will bring about chang e . Simultaneously, large numbers 
of teachers will be persuaded to behave differently, because different 
demands will be placed on them. 

Another by-product of a comprehensive system of accountability 
that is attractive to teachers will be a greater sharing of ideas. 

Very little has been done at the teacher level to create a bank of 
successful techniques. It's not be denied, of course, that we have 
grandiose schemes, naster-of-ar ts degrees in teaching, and lengthy 
courses. But these are all a bit removed from the firing line, and, 
in consequence, we never hear of — 01 from — the teacher out there, 
somewhere. The teacher who, ordinary enough most of t lie time, proves 
to be absolutely brilliant for just three lessons a y^ar. Three 
lessons in which she develops certain concepts better than anyone 
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else. I'd like to hear from her, and so would most other teachers, 

To develop better systems than we now have* we must pull together 
what is known out there — and use it. 

This suggests, of course, that an essential part of any 
system of professional accountability is the development of a model 
of what constitutes competent practice. Competent practice is not 
necessarily related to some particular performance result. It would 
be unwise to evaluate a doctor, for example, on the basis of the 
number of patients who die while in his care. If the doctor concerned 
is a cancer specialist — but the difficulty is obvious. Here the 
question of competent practice may have more to do with whether he 
prolonged life for a time, or relieved pain. 

So what is missing in our field o .* education, and must be 
developed in conjunction with the accountability movement, is a nodel 
of what a competent practitioner does when faced with a particular set 
of problems. 

Speaking of problems brings to mind some that exist with 
three currently popular ideas. These ideas — vouchers, performance 
contracting, and school decentralization — all seem to possess either 
basic flaws in the reasoning that promotes them, or in the manner in 
which they are being promoted. Hitherto, J have been talking about 
accountability mainly in connection with its impact on, and concerns 
for, one segment of the educational community — teachers. But the 
three ideas that I've just mentioned should be of concern to all of us, 
because they can be serious obstacles to the development of a true 
accountability system. 

First, vouchers ; which are being proposed as a national answer 
to providing accountability by offering a choice to the consumer — the 
student or his parents. It might be more accurate to say "the semblance 
of choice," because no one seems to have considered the implications of 
a nationwide voucher system. So let us consider them, and to make 
things a little simpler ve won't talk about the whole country, just New’ 
York City -- much simpler. 

Let's suppose that just 50 % of the students decided they would 
go to private or parochial schools in the future. That's a small natter 
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of 600,000 youngsters* Their decision would set off a chain of events, 
resembling nothing more than a child's game of ‘'Ring Around the Rosie. ,r 
With the public schools half-empty, half the teachers would be fired. 
Neighboring schools would be consolidated for efficiency and economy. 
Surplus buildings would be closed. The private institutions, besieged 
by 600,000 youngsters waving vouchers, would urgently need buildings, 
teachers, textbooks, and materials. And the only readi ly-available 
source of buildings, of 30,000 needed teachers, would be those closed 
public schools and surplus teachers who are out looking for jobs. We 
have come full circle: The same children, in the same schools, with 

the sane teachers* The great innovative voucher program has accomplished 
only one thing — it has removed responsibility from the government, 
because the schools are now private, not public. 

Those who would drastically limit the scope of a voucher 
program in order to avoid these problems must necessarily turn the 
program into one available only to the elite few — a program hardly 
worthy of national debate and national support. 

So much for vouchers. On performance contracting I want to 
start with the statement that, in a field as complex as education, 
there can be no guarantee of performance. The position is similar 
to that in other complex fields: a doctor or a lawyer cannot 

guarantee performance. If they did, they'd run the risk of being 
jailed as quacks. Perhaps those who purport to guarantee performance 
in education should also be jailed for quackery. 

The second problem with performance contracting was fore~ 
shadowed by my call earlier for a model of what constitutes competent 
practice. Performance contracting moves us away from real account- 
ability, away from analysis of what a competent practitioner should be 
doing, to consideration of a specific end product -- away from the 
process which the competent practitioner engages in to the product, 
which depends on many factors not within the control of teachers or 
schools . 

The next argument against performance contracting is that it 
seems to oversell an underdeveloped technology. I recommend to you a 
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very fine book by Anthony Oettinger. "Run Computer Run" is a thorough 
analysis of the state of educational technology today. Like Dr. 
Oettinger, I am hopeful that eventually we shall acquire very 
sophisticated technology. I am not against technology, ve need it, 
and we should develop it. 

But I am opposed to the manner in which the technology of 
performance contracting is being promoted. Performance contractors 
are behaving and talking as if a technological answer to all problems 
is already available. It isn’t, and these companies should admit that 
they are trying to develop such a technology and need the children in 
today's schools to do it. That it is only a try, and not a cure for 
today's ills. Anything less than such frankness smacks of deception. 

My fourth objection concerns the special motivational 
devices featured in most performance contracting programs. Radios, 
baseball bats, and green stamps are among the goodies being used. I'm 
not all that "holier than thou" about such things. I tell my son that 
if his report improves, he can have a new bike. We all use this 
approach, and there's no question that such rewards play an important 
role In our family life and our society. So we can’t say that 
rewards must never be used, but we must ask some serious questions — 
because no one else seems to be doing so. 

What happens to the student after he leaves tie motivated, 
reward-oriented climate of the performance contract classroom and 
returns to a regular' class? Does he refuse to learn? Does he fail 
to learn? Does the use of motivation in one room — which is not 
available to teachers elsewhere — create learning in one place and 
destroy it in another? And what happens next year, when the motivational 
goodies are withdrawn? I don't know the answer to these questions, and 
I suspect that no one else does, either. And because we don't know the 
answers, it is incumbent upon anyone who uses this type of reward system 
to build an analysis of it into the research design for his nrogtam. 

Finally on performance contracting, I .suggest a case of false 
packaging. I've already touched on the impossibility of guaranteeing 
a specified result, or level of performance. We are, of course, 
confronted with suggestions that this can and will be done. But what 
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we ars actually presented with is a non-guarantee . That is, it's not 
the student's performance that is_ guaranteed, but the contractor's pay- 
ment that is not guaranteed. 

We have even been oversold on the idea that the contractor 
doesn't get paid if the st. dent fails. That just isn’t true in the 
overwhelming majority of contracts. In fact, the contractor receives a 
succession of payments: When he signs; when he moves the hardware in; 

again at the halfway point; leaving only a fairly limited amount which 
he does not get if the children fail to succeed. In addition, many 
contracts absolve the company from responsibility for youngsters who 
fail to show up for the progi cm a certain number of times — usually 
fairly small. So it is that we have in the Bronx a program with a 
tremendous amount of absenteeism, and the companv stands to collect 
on the very students for whom the program was designed. 

So the company gets paid a good amount whether or not there 
are results; it gets paid for ( i truants and dropouts; and it can 
also profit from a well-known char, teristic of the standardised tests 
so commonly used today. I refer, of course, to errors of measurement. 
The simple fact is that if you tested a group of students today and 
again one month hence — having given them a vacation — 25% of that 
group would make, or appear to make, one whole year's progress in that 
short month of vacation. If you paid the company for that group and 
repeated the cycle, at the end of another month the company would again 
be eligible for payment on another 25% of the remaining students. Non- 
guaranteed payments begin to look more like a mirage, I think. 

I won't spend any time on the third obstacle to account- 
ability -- school decentralization. You all know what is suggested 
and I am more concerned with calling attention to what seems to lie 
behind theae three proposals: abdication, or evasion, of responsibility 

— or should I say accountability -- by the U.S. Government. 

In the last decade, w k have seen parents, teachers, 
administrators, labor unions, ar.d civil rights groups marching on 
Washington to demand mo-e money for education. Last year, the President 
suffered two major defeats when his education vetoes were overriden. 
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The pressures are obvious and insistent, and the Administration is 
seeking ways to silence these clamoring voices. So I think these 
three proposals represent a national strategy for reducing the 
accountability of the U.S. Government to our school systems, our 
parents, and our students. In each case, when the voices cry. "Our 
children are still not learning," as well they may, the Government 
will have a set of ready-made answers available. "You decided on 
the school; choose another if you don't like it." Or, "So get 
another performance contractor." And, of course, "It's your Beard 
of Education; you elected them. Elect another lot." 

In all, a strategy to reduce accountability by creating a 
phoney image of consumer choice. 

In reality, a strategy designed to take a major American 
institution, which has led to a good deal of social mobility and 
equality of opportunity, and to throw it away on a series of 
political gimmicks. These gimmicks should be rejected, for unlike 
many educational experiments which can be tried and, if they fail, 
be rejected — these experiments which reduce the commitment of 
government to education and which move the schools from the public 
to the private sector are, like experiments with hard drugs, 
irreversible. Our public schools, with all their faults, are worth 
keeping, and their improvement will com*' not from gimmicks \ ut from 
the same type of slow, painfol, unrestricted, free, scientific inquiry 
that brought other areas of human concern into the modern world. 
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PUBLIC EXPECTATIONS 
I 



Wilson C, Riles 



A Gallup Poll in 1970 found that 67 percent of the people 
contacted believed teachers and school administrators should be 
held more accountable for the progress of their students. In the 
rise and fall of fads, this percentage should increase for some 
time to come as word gets around about the "magic" of acccuntabili ty . 
Following much more slowly will be the practice and fact cf 
accountability, and hopefully by the time the public switches tracks 
to another destination, accountability will have settled permanent- 
ly into our school system as a commor.-sens e measure without the 
guise of a panacea. 

Perhaps it takes these public exigencies to spur needed 
change in the education profession. The profession has the 
peculiar quality of being able to reform others without being able 
to reform itself. All the public is asking, after all, is the 
same high standards of respon ibility with the public monies that 
they demand in the manager ■ 11 1 of their own private affairs. 

The source of the current interest in accountability is 
fairly well known: school needs have outrun scln *"nnds , prior- 

ities are having to be sc t, and the public is no longer satisfied 
with allocations that do not clearly reflect the ' rilies. The 
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public feels that with school budgets, as with their private budgets, 
there ought to be reason and priorities, the expenditures should be 
balanced, and you should have "something to show 11 at the end of the 
process. Moreover, wherever possible the factors involved should 
be reduced to cold hard facts — just as with the space program, 
just as when an individual buys an automobile or makes a busing 
investment. No emotion, no poetry, just cold hard facts. 

Whether the analogies are directly transferable to educ- 
ation or not may not be as important as whether the public thinks 
they are transferable. Because the public r s bel.ef in the 
similarities between running a business and running a school may 
become the public T s expectations in accountability, those expecta- 
tions may be what educators will have l< contend will: '.ost. 

In my own state there are growing demands for regular 
evaluation of teacher performance with a prepared check-off list. 

There are editorials proposing to quantify everything from the 
bus driver’s free time to student attitudes in art. And there are 
people wanting to reduce all values to a square-foot or a cent by 
cent expenditure. The reaction of the profess ion, it seems to me, 
can be one of resistance and counterclaim; which i don’t think is 
really a plausible reaction at all. Or it can be one of greeting 
the new interest as a welcome enthusiasm for progress, with an 
invitation to the public to help implement the precepts. This, of 
course, is the position I think the education profession should 
take. Indeed, I cannot imagine how accountability would work 
otherwise. Accountability is essentially a partnership venture. 

I believe the public's expectations for accountability — 
whatever they may be -- should be meshed with the public’s partic- 
ipation in the accountability process. If this occurs, then for 
once the hopes and the facts would be the same. Let me be specific. 

I view accountabi 1 ity as a process of setting goals, mak- 
ing available adequate resources to meet those goals, and conduct- 
ing regular evaluations to determine if the goals are met. Funda- 
mental to this process is that there exist an adequate "data bank" 
of information from which viable options can bo determined. The 
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researchers and che state departments of education should provide 
this. From the available options, then, goals can be determined. 

In the goal-setting stage, the broadest possible spectrum 
of die community should be brought together to make the decisions. 
The process should be comprehensive and cohesive, involving stu- 
dents, parents, teachers, administr tors, boards of education, 
legislators, and the public at large. Once goals have been set, 
the necessary resources can be allocated. The public will know 
what is needed from the data bank. They also will know if they 
do not allocate the amount needed, then it is unfair to expect 
the schools to meet the assigned goals later. 

Finally, there’s evaluation — comprehensive, in depth, 
and accurate. This cannot be a one-score test evaluation, but 
must be an ongoing, regular evaluation that is diagnostic as well 
as comparative, that accounts for process as well as product, and 
that is principally geared toward improving instruction for the 
individual student. Moreover, the evaluation results should be 
translated into terms that are clear and easily understood by the 
lay public. A regular ’’state of education" message would seem to 
be a must, and the terminology used should be such that the options 
available to the public are clearly laid out. Then the account- 
ability process can begin again. 

The thrust of this accountability system would be that 
the taxpayer is never asked to support inefficient schools, and 
that the people have a regular meaningful assessment of the quality 
of education in their communi t ies . If the people have participated 
in establishing the goals and have a significant voice in the 
assessment, then there is a higher chance that their expectations 
will bo geared into reality. Otherwise, with only an outside 
knowledge of education, I see no reason why the public shouldn’t 
expect accountability to recast our schools into slide-rule 
perfection. If the latter persuasion takes sway, we can expect 
some awkward moments. 

Right now in Los Angeles, serious thought is being given 
to decentralizing the city district into a dozen mini-districts. 

The reason is that many people feel local schools should be made 
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more responsive — and hence accountable -- to residents. But at 
the same time, and for the same reason, the Governor's Commission 
on Educational Reform is proposing that California abolish the 58 
county superintendent positions in favor of 15 state department 
of education regional offices. The problem is, no one lias clearly 
determined the influence of district size on efficiency. I know 
of small inefficient districts as veil as small efficient ones. 

We still are operating largely on hunches. 

Accountability, too, if improperly handled, can bring 
some self-defeating results when paired with the public expectations. 
Administratively, the bookkeeping could be overwhelming with 
ineffectiveness that could pique the public anger. Or anger might 
come from the presence of outside research teams at the local 
school, evaluating the neighborhood 1 s children. There's a very 
strong possibility a parent may want rigorous accountability 
standards used on every child but his own. Or that accoun tabili ty 
results will be used more for comparative than diagnostic purposes. 
This is the case now with California's statewide testing system. 

It serves more as fodder in political and legislative wrestling 
matches than it does as a source of improved instruction for the 
child . 

No doubt accountability does lend itself to becoming a 
battlefield for the "experts." Facts and counterfacts always seem 
to be in plentiful supply, and everyone can garner up an arsenal 
of experts to authenticate his case. Too, the critical process 
of interpreting raw data to the public is particularly susceptible 
to distortion, and it is the rare reporter or administrator who 
can penetrate into the mysterious and protected reserve of the 
statistician. 

Politically, of course, there is a danger that school 
board elections might be won or lost on the basis of approximations 
and estimates, wnen in fact the figures nay be generalities at 
best. Or legally, there may be these questions: How much pow T er 

can be farmed out to private performance contracting groups; do 
those groups have to use state-certified personnel; who is liable 
for quotas set but not met? 
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Educac ionally , there nay be- fears of a new rigidity intro- 
duced into the system, particularly if everything is quantified and 
we have a series of "five-year plans." With accountability as the 
byword, there may be some tendency to discourage courses that don't 
lend themselves so readily to quantifiable measures: or to discourage 
services — such as counseling or health — that may not have 
quantifiably ascertainable results. And finally, in our rush for 
certitude, we might snuff out those variables in education that make 
for human creativity and imagination. 

These possibilities are some of the reasons why I believe 
that if the public is not brought in on the process, they will ham- 
mer at it from the outside and eventually establish procedures 
devoid of the input of the profession. 1 cc not believe, for 
instance, that someone who is brought into the evaluating process 
will demand that all values be reduced to numbers* Nor, to the 
contrary, will they any longer claim that no values can be reduced 
to numbers. Instead, 1 believe they will understand thaL some 
things can be quantified and that others cannot; and that these 
things that can be quantified should be quantified so that those 
things that cannot will have greater play. 

Creativity and innovation are challenged today more bv 
inefficiency and lack of direction than they are by systems 
analysis. Freedom is a function of your optics, and today our 
options are precious few. Far from: engineering ran out of education, 
I believe accountability is an attempt to bring man bach in. V'bat 
va have been squeezed out by is our own ineptitude and archaic 
methods that have kept us so busy we haven't had time to be human. 

Thus, in surimary, it seems to me our principal t a k is not 
to fret about whether the Gallup Toll registers a rise or decline 
in public expectations about accountant 1 i t v , but rather we should 
get busy working directly with the public to make accountability 
a functioning process for improving quality in our schools. Then 
the expectations will more likely approach what is truly possible. 
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PUBLIC EXPECTATIONS 
II 



H. Thomas James 



Popular concern about the performance of educational 
institutions is not a new phenomenon. One could perhaps explore the 
concern of the French government early in the last century over the 
rapid rise of Prussia as a powerful national and industrial state, 
the French decision to employ Victor Cousin to study the Prussian 
education system, and the subsequent transformations of French and 
American educational institutions traceable to that model. There 
are also local illustrations, such as the discontent in Quincy, 
Massachusetts, which led to the school committe f s decision in 1876 
to conduct the general testing of the school children themselves, 
and to the subsequent revolutionary reorganization of that school 
system that brought over 30,000 people to Quincy in a subsequent 
three-year period to view the remarkable results. I’m sure other 
historical illustrations can be suggested as examples of the classic 
pattern of political storms gathering about educational institutions 
which led to reforms, such as those experienced in many big city 
school systems around the turn of the century. Again late in the 
1930 1 s, concern was in the air and reforms we r e in the making but 
aborted, perhaps because of the distractions and dislocations of 
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World War II, thus leaving serious problems still to be resolved 
through more extensive reforms than have yet been attempted. 

The first surfacing of a truly national concern about 
conditions that have led to the presert state of crisis in American 
education appears to me to have occurred in a meeting of educators 
in 1946 that included James B. Conant and Roy E. Larsen, which led 
to the establishment of the National Citizens Commission for the 
Public Schools. The Commission (later Council) promoted citizens 
support of local schools, and citizen interest in programs and 
problems of the schools, without taking issues on such professional 
matters as curriculum and methods of teaching. Its existence, 
activities, and support provided state and local forums, and frame- 
works for discussion, that were enormously helpful in aggregating 
demands arising out of discontent with schools and in shaping a 
strong political drive for funds to support the costs of rising 
school enrollments following World War II. The public relations 
approach that they used during the 1950's taught teacher associations 
a lesson useful to their purposes that has helped maintain their 
sophisticated efforts for support of schools long after the 
Commission's major effort ended. 

The Ford Foundat ion ' s Fund for Education, which supported 
the Commission, moved on from the Commission's essentially public 
relations approach to problems created by rising enrollments, and 
began focusing attention on qualitative problems related to curric- 
ulum* teaching methods, and administration, and laid the groundwork 
for much of the governmental efforts that followed through the 
National Defense Education Act, the National Science Foundation, and 
the Elementary and Secondary Education Act of 1965, the latter mark- 
ing the first serious congressional -effort to compensate for racial 
and religious discrimination in state and local administration of 
schools. Other voluntary efforts, including Educational Testing 
Service, and the Education Commission of the States, which is now 
administering the national assessment of education, have added 
substantial capabilities for diagnosing the current ills and perhaps 
have aided in shifting attention from fiscal problems, administrative 
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organization, curriculum, and teacher training, to where the deepest 
popular concern has always been, namely, with the effect of the 
school on the individual child. The widespread interest in the 
current spa* of apocalyptic writers, who advocate destruction of 
the school as we know it, may signal a popular readiness to consider 
more fundamental efforts to improve the schools than anything we 
have seen since the Quincy New Departure. 

The popular uneasiness about the schools is further 
evidenced in the xcids acceptance of certain terms, such as M Johnny 
can’t read" in the 1950’s, "the pursuit of excellence" in the 1960 T s, 
and most recently, "accountability," a term Leon Lessinger popular- 
ized that has since appeared often in Presidential and other political 
references to education. Because of its popularity, and the new 
meanings being read into accountability in the last year or sr, many 
individuals and organizations are seeking ways to deal with it. 

I will make a brief attempt in the next few paragraphs to 
explore what definitions I have been able to find for the term 
"accountability" and then express what I can discern of the appeal 
each has for the public ' 1 the educationcl context of today. I will 
make no effort to define what Leon Lessinger meant by it, because 
I am sure from observing him over the years that he, like the Queen 
in Ali ce in Wonderland , used the term to mean exactly what he 
intended it to mean, no more and no less, I am sure, also from 
observation, that his intentions vary from time to time, as alsc, no 
doubt, do those of politicians using the term. 

In summary there appear to be at least six general catego- 
ries of meaning related to accountability as follows: 

1. Attributable, the assignment of cause, placement of 
blame, accounting for, as in sources of change, fix- 
ing of rt?sponsib x li ty. 

2 . Predictable, divineable, calculable, accountable in 
the sense that a contract makes some part of the 
future foretellable , ant icipat ab le , foreknowab le , 
promised . 
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Intelligible, comprehensible, discoverable, under- 
standable, fathomable, conceivable, accountable in 
the sense of being easily understood, unequivocal, 
unambiguous, unconfused, "in plain English." 

4. Explainable, interpre table , deducible, capable of 
being inferred, describable, definable, translatable, 
demons trab le . 

5 . Liable, answerable for blame, bound to duty, imexempt 
from responsibility, answerable for obligations. 

6. Subject to audit, taking of inventory, balancing of 
accounts, "be checked up on," have books examined, 
be verified, particularly from the standpoint of be- 
ing economical, thrifty, prudent, provident, and 
demonstrating good management or stewardship. 

As these meanings associated with accountability are 
examined, and doubtless as others will emerge as we study it, one 
begins to see why the term "accountability" has caught on so quickly 
and appealed so broadly. For the perennial critics of the schools, 
it provides a convenient shorthand to summarize all of the major 
charges they have leveled at schoolmen over the years as irrespon- 
sible, unpredictable, incomprehensible in the "bafflegab" or 
"educat ionese" they speak, incapable of either explaining or 
demonstrating what they are doing, never held liable for their 
failures, and at schools as improvident, and badly managed. The 
term appeals also to the parent who would like to fix responsibility 
for his child's education, anticipate his progress, understand what 
his teachers are talking about, have them explain and demonstrate 
what they do and what the parent can do to help, know that teachers 
at? committed to their obligations and will be answerable for biaire 
when it is deserved, and finally that the whole operation be audited, 
both as to the fiscal as well as to the educational record. 

Yet from the standpoint of the school administrator, as 
the responsible head of the school , he’s left virtually defenseless 
in all categories except the sixth, and these can depend only on 
his records of fiscal management, which are badly in need of extensive 
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reorganization (a subject I hope we can explore in due time, for 
here, at least, is an area where the Chief State School Officers, 
MSA, ASBO, NEA, and the Office of Education, among others, have 
demonstrated in each of the past six decades a willingness to 
cooperate). The results of the teaching act are measured over long 
periods of time in which many teachers are involved with a given 
child, and the assignment of cause for an individual failure <unonr 
such diffuse contributions is virtually impossible under existing 
arrangements for schooling. Despite • tudit_s such as Benjamin Bloom's 
that argue the feasibility, few teachers willingly predict a child's 
future performance in school or elsewhere, nor will they normally 
agree to guarantee performance levels. The typical superintendent 
rarely can admit to understanding all his teachers, let alone 
guarantee that they will understand each other, or be understood 
by the public. As for explaining or demonstrating what goes on in 
a classroom, teachers, like churchmen, find mystique more helpful, 

The question of liability rarely arises, for the contractual respon- 
sibilities are not specified in terms other than being in certain 
places at certain times and "teaching” specified pupils. Teachers 
have successfully resisted attempts to audit their performances ;u 
terms of the behavior of children, so only ihe fiscal side of the 
school's operation is audited. 

From the standpoint of the larger governmental structure, 
again most of the existing require? nts tor accountability fall in 
category six, dealing with the fiscal operations and with counting 
of pupils and personnel in specified categories. Only in the case 
of malfeasance, strictly limited by statuu in its definition, will 
government search out and place blame. The ml" f u r ure-oriented 
expectation for performance from the standpoint u.! higher adminis- 
trative echelons ivS that progress through grade- shall eq ’ate roughly 
with age groups. That profession *1 discourse about schools he 
intelligible, or actions related to school personnel be explainable 
or demonstrable, seems not to Have concerned school governance at 
any level, and matters of liability are confined to narrowly defined 
"causes" rarely remotely related to the performance of students. 
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Other agencies provide limited remedies. Private schools 
provide an alternative to parents who are convinced that blame for 
a child's performance rests with the public school. Proposals now 
under discussion to broaden the availability of that alternative, 
such as the voucher plan, apparently will be tested in at least 
some limited ways in the near future and the results will merit 
careful study. Predictability is offered by firms seeking perform- 
ance contracts, and these, too, will merit our careful study. Some 
of the very best of our schools of education are drawing disciplined 
minds into the study of educational phenomena, structures, and 
functions; out of these efforts are coming the most hopeful signs of 
a developing pedagogical discourse that will be comprehensib le not 
only across disciplines but to the literate layman as well. I hope 
that we can give some systematic attention to this development, and 
perhaps illustrate the dimensions and depth and possibilities for 
extension of that discourse. 

The explainab i li ty and demons trab i lity of processes and 
practices in schooling are perhaps the most puzzling aspects of 
accountability. The notion that any teaching method that works is 
good teaching is so pervasive among American educators and so widely 
accepted popularly that we have made little progress in this century 
in developing sound theories in pedagogy. The problem seems to be 
that any innovation, tried by a dedicated teacher and carried through 
with passion and commitment, works once. V, T e thus have developed an 
incredible array of methods, proven by the creator, which not only 
do not advance our theoretical constructs but actually get in the 
way of school improvement, because they often create distractions 
and failures when others attempt to apply them to practice. One 
would hope that more scholars would recognize an opportunity to 
examine this perennial problem, of interest not only to American 
educators, and to say something significant about the contributions 
of the remarkable range of experimentation in the 60 r s to pedagogical 
theory . 

One encounters much discussion, extending back over the 
past several decades, about liability for performance by teachers, 
notably a system of sanctions that w T ould reward high performance 
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(merit pay) though there seem to he few proposals seriously put fur- 
ward to penalize bad performance, short of dismissal. Some teacher 
groups are exploring the possibility of entering into performance 
contracts as a countermeasure to contracting with firms, and we can 
perhaps see the beginning of a transformation in teacher contractual 
relationships in these efforts. A distinguished New York attorney 
and Regent, Max Rubin, raised an interesting point recently with 
A1 Shanker in a small group I was with: If teacher contracts 

continue to become more specific, may not the employing agency 
eventually be in a position to hold the union liable r or unsatisfac- 
tory performance? I am sure others in the legal professi n will 
show interest in exploring the feasibility of this idea. 

We have a long tradition of auditing firms providing 
services for the fiscal audit. University professors have tradition- 
ally done management and other types of surveys. More recently, 
management consulting firms, notably Arthur D. Little, Booz Allen 
and Hamilton, and Cresap, Paget and McCormick, have moved in on this 
type of survey, and many new firms are in the field gradually taking 
over the university field service function. Leon Lessingev has 
frequently discussed an r 'educa t ional audit” and the significance of 
this type of service to schools is likely to be tested in the next 
few years . 

The current interest in accountability in education is 
likely to have profound consequences on schools, for it raises the 
inevitable question, Accountable for what? 11 To answer this ques- 
tion requires the specification of goals in education. Those who 
use accountability as a lever for change, particularly those firms 
that seek performance contracts, are accustomed to analyzing pro- 
blems through the use of mechanistic models that have proved useful 
to engineers, and more recently to economists and business firms. 

As we begin to apply those models to education problems (a subject 
1 explored at greater length elsewhere'*), we find that the first 
step is specifying our goals. "We are, after all, attempting to 
recreate our social world, and especially our schools, to fit a 
model of our invention. We reason that, since we have created con- 
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plex machines, we can now use the laws we have derived from that 
experience to reconstruct our social institutions. In that effort 
we may violate two laws of logic: (1) We may apply our mechanical 
models to concerns too broad to be encompassed when we fail to 
perceive the proper scope of the human condition, and (2) we may 
apply our model to inconsequential ends when we analyze less encom- 
passing statements of human aims. 11 ^ 

In our first efforts in performance contracting in schools 
we seem to be erring toward the violation of the second law of logic 
applying our model to inconsequential goals in education. If we 
teach the child to read, and to count, the people will ask, as Plato 
did, why haven't we also toiight him to be virtuous? 

We have been notably unsuccessful as a society in this 
century in stating our aims of education. To face the prospect of 
being driven by circumstances, created as casually as by acceptance 
of the concept of accountability, to set trivial goals for our 
educational institutions, is appalling. A quite contrary course 
seems indicated, rather to dare to set our goals to fit our broadest 
percept i. .. . the sc^pe of the human condition, and to challenge our 
model-builders to reach toward them, and to be critical of their 
failures to reach them. 

In the remainder of this paper I shall touch briefly on 
the major aims of education that I perceive as pervasively accepted 
in the historical documents and contracts >'f our society, and that I 
believe are present yet today in the broadly a* copied expectations 
of our people. It would seem to me to be a propitious time in our 
history to test our consensus on these broader goals, if only to 
alert us to the dangers of becoming distracted by the increasing 
triviality of current efforts to state the aims of education. 

For the early founders of schools in this country the aims 
of education were, quite simply, piety and civility, two forms of 
behavior extensively discussed in the education literature of the 
16th and 17th centuries. 

Ihe concern for piety meant that children must be taught 
to read in order that they could study the Bible and acquire thereby 
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religious faith, spiritual mindedness , temperance, purity, righteous- 
ness, and charity, and thereby join the elect, those to be saved 
after death, the children of Cod. 

The concern for civility involved teaching of good manners 
and deportment, prudence, courtesy and thoughtf ullness , affability, 
gentleness, urbanity, tolerance and graciousness toward others. 

The educational literature of the 18th century reflected 
trie growing impact of the Renaissance on the popular consciousness 
in its addition of Lh n pursuit of knowledge as an aim of education, 
and reflected also the growing interest in the political philosophers 
who emphasized education as a necessary ingredient for a self-govern- 
ing society. The new Congress in 1787 combined in their preface to 
the Northwest Ordinance their transformation of the earlier aims of 
17th century education and the additions of the 18th in the opening 
words of that Ordinance: ’'Religion, morality and knowledge being 

necessary to good government, schools end the means of education 
shall be forever encouraged." The curricular materials dealing with 
morality persisted late in the 19th century, notably in the McGr.ffey 
Readers . The extension of knowledge, especially in the sciences and 
in mathematics, increased the subjects of study and began the con- 

i 

fusion, still evident in schools, arising from, the assumption that 
learning facts is educative. The emphasis on good government made- 
much of the fundamental values of our society having to do with 
liberty and equality, two values that, taken together, mark a pro- 
found and continuing dilemma in America. 

In the 19th century the industrial revolution was reflected 
in an additional expectation for the task of the schools, that: they 
teach children to be productive. Productivity, with its connotations 
of f rui t fulness , abundance, creativity, inventiveness, ingenuity, 
acquisitiveness, gainful employment, earning, saving, and investment , 
is perhaps best epitomized in the Morrell Act ot 1862 establishing 
the land grant colleges, and in succeeding efforts to encourage 
vocational education and manpower training that are continuing even 
into deliberations of the Congress now in session. 
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Our own century began with rather general acceptance of the 
aims of education from the past. Piety, with its moral basis for 
action, if abstracted from the morass of quarrels among sectarian 
religions that have plagued the concept from Reformation days, prob- 
ably is still a broadly acceptable aim for education among our people 
today. Certainly the concept of civility is still with us, if evi- 
denced only by the persistent cries for its restoration to discourse 
and relationships in the present. Certainly the pursuit of knowledge, 
the intellectual, or, to use the currently popular tern, the cognitive 
aspects of education, is still broadly acceptable as an aim. Concern 
is now being expressed by both educators and youth for greater emphasis 
on the emotional or affective aspects. Our growing dissatisfaction 
with the quality of discourse almost certainly portends greater 
emphasis on the concept of civility as it was defined in our earlier 
history, or as it may become redefined in whatever transformation of 
the concept we can achieve in our time. Concern for good citizenship 
as an aim of education persists also, though broadened unquestion- 
ably from narrow nationalistic concerns to a deeper social conscious- 
ness, a concern for the environment, and recognition of the need for 
good government and good citizens for the whole world. Certainly the 
concern for productivity persists broadly among our people, though 
here the dissenting voices are heard so loudly, especially from the 
younger age groups, that we can assume some major transformation of 
this concern is imminent. 

To describe a man or a society as pious, civil, knowledge- 
able, self-governing, and productive, using these terms ns we find 
them defined in their best traditional sense as aims of education in 
our society, is to endow both the man and the society with most of 
the cardinal virtues, but not all of them. In the last two decades 
we appear to be exhibiting a deepoi concern for justice in the 
distribution of social and economic benefits than has been made 
specific in our earlier curricula; evidence that the lessons have 
been well-taught is emerging, most markedly in the interests and 
actions of the recent graduates of our schools. Our great 
unfinished task is to find some way to ttach nope, for in this 
virtue our current graduates seem sadly deficient. We are finding 
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sympathetic listeners also to the proposition that schools might be 
conducted in more humane ways, might even be happy and joyful places 
in which to spend a significant part of one’s life, a possibility 
that seems not to have occurred to tho5;e who earlier shaped the 
American schools. 

These, then, are some of the traditional aims of education 
in our society, with some speculation on those emerging. One can 
argue that they are global concepts derived from philosophy and 
religion, and therefore of little use in an age that seeks to define 
its educational objectives in behavioral terms. My reply would be 
that there is a rich literature, which doubtless can be further 
enriched, that offers ample opportunity for selecting remarkably 
broad sets cf behavioral objectives related to each of the traditional 
aims. I think no one can seriously argue that any one of the concepts 
is irrelevant in our time. 

It was with these aims for education in mind that state 
legislatures enacted the laws that established the state school 
systems through the 19th and into the 20th centuries. It Is in 
terms of these aims, or synonyms or euphemisms for them, that the 
larger controversies and criticisms of the schools are phrased. We 
can stir national concern about the assertion that Johnny can’t read, 
but when citizens meet in their local communities to discuss that 
assertion, the discussion shifts to Johnny’s manners, his dress, the 
length of his hair, nis morals, his religious attitudes, his values, 
and what lie’s thinking of doing with his life. And it is in terms 
of these aims that the programs, the faculties, and the students of 
schools of the future will be judged. We need to develop new 
standards for measuring the performance of our educational 
Institutions and for reporting on that performance and many people 
of good will are going about that task in many ways. The plea I 
offer is that we attend not only to the minutia but that ve attend 
also to shaping standards and criteria for judging how well we 
achieve the grand aims of education which are certain to persist in 
the minds of our people. I wish you nil well in the task ahead, and 
hope with you that we can find ways to restate the aims of education 
more attractively and more in line with our great tradition. 
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THE ROLE OF EVALUATION 



Henry S. Dyer 



AND VICE VERSA 

Three events in the history of American education illuminate 
some of the more important roles that evaluation must play in any 
system of educational accountability. 

I 

The first event occurred in 1647 when the Great and General 
Court of the Massachusetts Bay Colony enacted what the his Lory books 
refer to as the Old Deluder Satan Law, ^ This, you will remember, was 
a law that sought to foil the designs of the devil by insisting that 
every child in the Colony be taught to read and write. It held each 
town a ccountable for providing this instruction out of its own funds. 
And it backed up its mandate with an annual fine of five pounds to be 
levied on any town that failed to comply. 

One reason, no doubt, that the Puritan Fathers were able to 
get au ty with this high-handed infringement on local autonomy was that 
there was general agreement in those days on the ends and means of 
education. All children must be taught to read so that they could have 
direct access la the Scriptures and thereby have an outside chance of 
avoiding eternal damnation. One of the major problems in education 
these days is that peaple are not all that clear and convinced about 
the ends and means of education. 
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One reason for tins staLe of affairs has been suggested by 
Lawrence Cremin: 

'b . .too few educationel leaders in the United States are 
genuinely preoccupied with educational issues because 
they have no clear ideas about education... They h.ivi 
too often been managers, facilitators, politicians in tiie 
narrow sense. They have been concerned with, building 
buildings, balancing budgets, and pacifying parents, but 
they have not been prepared to spark a public .‘bate about 
the ends and means of education. 1 ' 

Another reason for I ho ; u.izlncss about ends and nears is u.al 
educational goals, as commonly formulate* by educational phi lrsop! < ’ s , 
have tended to be cast in such sweeping generalities and remo o idesls 
that they have left school \ eople at a loss to use them Meaningfully 
for assessing the actual ongoing operations of their instiLut t oru 
This statement is not intended to denigrate the efforts of educational 
philosophers. Their ideas are a necessary, if neglected, ingredient 
of the process by which usable goals can be defined and applied in 
ccn’rcte instances. But they are only the beginning of the process; 
the gulf between the expression of educational ideals and any practical 
measure of their realization is so wide- and deep that few i ' any 
working educators have been able to fLnd their way across it. 

The educational oratory speaks of goals like "self-f ulf i 11- 
men t, 11 "responsible citizenship," and "vocational effectiveness;" the 
assessment of school efficiency in specific cases usually depends on 
such measures as retention rate, college-going rate, average daily 
attendance, and performance on reading tests. Whether there are any 
rational connections between the numbers and the slogans as a matter 
that is rarely considered. The assumption seems to be implicit, for 
instance, that the longer a youngster stays in school, the greater will 
be his chances of self-fulfillment; or that the higher his reading 
score, the more likely t hat he will become a responsible citizen. But 
such assumptions are left largely unexamined, and in particular rases 
may be obviously wrong. In sliort , the answer to the ell-important quet • 
tion, "Accountable for what?" is left hinging in midair. 
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Therefore, one important and decisive role that evaluation 
must play these days in any educational accountability system, which 
is not designed solely to find scapegoats to assuage our collective 
guilt, is that of helping all of us sort out and eval u ate our educa- 
tional goals and objectives, so that we can begin to get some definite 
and agreed-upon ideas of where we want the schools to be taking us 
as well as our children, and what we think the priorities ought to 




Over the years there have been some promising efforts in 
coping with this problem of goal-setting at a practical level, but 
a lot still remains to be done if the community served by the schools 
is to become as deeply and significantly involved in the process as 
it ir.ust be if the notion of accountability is to make any sense at 
all in shaping education to fit the individual needs of the pupils 
as well as the needs of the troubled society that they are going to 
inhei it . 



II 

The next historical event, illustrative of another aspect 
of the accountability doctrine, in education, occurred nearly 300 
years after the enactment of the Old Deluder Satan Law -- in 1930 to 
be exact. This is a bit of personal history, for 1930 was the date 
when I had my own first traumatic experience of being held profes- 
sionally accountable as a teacher. I was in my first job teaching 
senior English. I had one particularly weak student whose parents 
were bound and determined that be should be shoehorned into a certain 
prestige college that I firmly believed was well beyond his capabili- 
ties. My principal gave me to understand in no uncertain terns 
that, for my part in this process, J was to be held a ccounta ble? for 
seeing to it that the boy passed the old-style College Board exam in 
English at a level that would make him admissible to the college his 
parents had chosen for him. The implication was that if the boy 
failed to make it, the renewal of my contract would bo in doubt. In 
short, \„y performance as a teacher was to be evaluated, at least in 
part, on how that student performed on that exam. 
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Back in the 30’ s the College Board exams -- unlike these of 
today — had passing scores which were defined in terms of performance 
criteria laid down by the examiners. Today, I suppose those old- 
fashioned exams of forty years ago, with all their presumed faults, 
would have probably qualified as "criterion-referenced tests." It 
is curious how history — euen in testing — seems to be repeating 
itself. 

In any case, what did I do to prove my accountability in 
that situation? How did 1 go about getting students to meet the 
criterion set up by that old-time criterion-referenced test in 
English? I did what many other hi gb school teachers were doing in 
those days. I crammed my students on all the old College Board exam 
questions of the preceding ten years, filled the kids up with canned 
themes so that they might appear to write profoundly , though pos- 
sibly a bit irrelevantly, on any topic that the examiners might 
dream up, and ground the standard literary classics into their heads 
until they v;ere thoroughly sick of them. 

By so doing 1 fulfilled my obligation and my contract was 
renewed. My weakest student passed the English entrance exam with 
flying colors. He was admitted in September 1930 to the college his 
parents had chosen. He flunked all of Ills mid-semester examinations 
in November 1930, and was fired shortly thereafter. By meeting my 
obligation under the narrow definition of teacher accountability 
then prevailing 1 had succeeded in preparing the student lo become 
a failure in college, 

Vhat does this episode suggest about the role of evaluation 
in an accountability system? It suggests that if the system is to 
work to the benefit rather than the detriment of the young, people who 
go to school, we must be continually observing and evaluating the 
side-effects and the after-effects of what goes on in classrooms. 

For if, by the processes we employ, we teach children to pass tests 
at the expense of learning to hate the subject in which we test them; 
or to hate the whole idea of learning, it seems to me we defeat the 
whole purpose of education and fail to be accountable to the student? 
themselves , 




The armamentarium of educational and psychological measure- 
ment contains a good many instruments of various types for evalua- 
ting students' attitudes coward learning, toward themselves, and 
toward one another. Admittedly, these instruments arc still pretty 
crude. The state of the art in the measurement of attitudes, values, 
and the like was summed up by David Krathwohl and his collaborators 
in these words in their book on educational objectives in the 
affective domain: 

"...we cite many techniques for appraising such objectives, 
but we are fully aware of the fact tiiat much must be done 
before the development or testing techniques in the 
affective domain will reach the rather high state of 
clarity and precision which is now possible in the cognitive 
dotna in . 11 

Nonetheless, if, as we have been saying all along, the schools 
are to be concerned about the development of the whole child, we had 
better make judicious but regular use of the best of these techniques. 

Be it noted, however, that such techniques should not , in my view, 
be used as a basis for evaluating the children themselves. They 
should be used, rather, as a basis for coming as close as possible 
to evaluating the full impact that schooling may be having upon the 
Jives of the children. insofar as schools fail to do their best to 
seek out this kind of evaluative information about themselves regularly 
and routinely they are failing to be accountable in any educationally 
acceptable sense of the word. 



Ill 

The third historic cate* in the development of the principle 
of accountability in education was April Lyb5 -- the date when the 
K lenient ary and Secondary .Education Act was signed into law. You will 
recall that clauses 5 and 6 In Section 205 (a) of the original Act 
provided that procedures be adopted for annual lv evaluating programs 
designed to meet the needs of educationally deprived children and that 
the evaluative data accruing from these procedures was to be incorpo- 
rated in annual reports f rum each local education agency to the state 
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education agency and thence to the Federal government , “* The purpose, 
of course, was to try to account for the Incremental educational 
benefits that the Federal dollars were buying, and it is this aspect 
of the evaluation/accountability equation that is unders Landat lv 
uppermost today in the minds of many taxpayers and llieir representa- 
tives on school boards and in legislative bodies. 

In view of the agonizing fiscal crises in so many school 
districts, this is, of course, a legitimate concern. It is a concern, 
however, that generally has overlooked the difficult problem of 
providing the needed evaluative information. The authors of one 
intensive study of the early functioning of ESEA have said that 11 when 
ESEA was in Its first weeks and months of implementations... the 
infrastructure of sys tematic program evaluation was either nonexistent 
or woefully primitive. ir ^ Anyone whe has kept up Vv T ith attempts to 
evaluate ESEA programs -- particularly Title I programs — in the 
last six years knows that this statement is still largely true, in 
spite of some noble efforts to lick the problem. At least part of 
the reason is that there are still nowhere near enough people out 
there in the school districts who know how to put a dependable and 
meaningful evaluation program together -- one that is capable uf 
genuinely and dependably relating educational benefits to educational 
costs, and this deapite numerous attempte to apply to the educational 
enterprise such appealing notions as cost-effect ivem-ss , planning- 
programriing-budget Ing , management information systems, and the like. 

Be all this as it may, it seems to me that the most impor- 
tant aspect of Section 205 of ESEA is not that it appeared to hold 
local school systems accountable for making educational expenditures i 
product: a measurable payoff in pupil learning. In point of fact it ‘ 
did nothing of the kind. It you read the original Act carefully, you 
cone to realize that all it called for was merely a rendering of an 
accoun tin g — an evaluation, if you will -- of what was going on i 
Title I programs and how 1 they were working. The big emphasis wa'i 
and still is, on objective and accurate annual reports on how the 
educational process is functioning on behalf of students and how muck 
money is being spent in the effort. This is a type of annual report 




C 



7 



1-7 



that had nevei- been produced before and, to my knowledge, has not 
been produced yet. Ve may know how much we spend on textbooks, on 
teachers 1 salaries, on busing, on food service, and so on, but we 
still do not know how to cost out a program in elementary school 
reading, or high school science, or health, or whatever, in such a 
way that we can actually isolate the costs of each program p_er 
and relate those costs to the children's growth in reading compe- 
tence, oi their love of books, or their physical well-being. 

When you put the problem of rendering an accounting in 
this way, you may well begin to wonder whether the problem, like 
that of squaring the circle, can ever be solved. It suggests that, 
in approaching the question of how to render an accounting of what 
is going on in an educational system, there is a real question of 
how far the accountability concepts that may be useful in the con- 
trol of industrial systems can be applied to school systems. For 
the production of learning and human development is hardly analogous 
to the production of soap or cat food or space vehicles. 

Moreover, the neasuremeat problem in each case is just 
about as different as it can be. In trying to achieve accurate 
measurement of the inputs and outputs of the industrial enterprise, 
one is concerned with making the human factors in the measurement 
piocess as small as possible, and in many areas the instrumentation 
for this purpose has become remarkably automatic and efficient. In 
the measurement of the cognitive.* and psychosocial functioning of 
students, however, the human factors are the very essence of what we 
are trying to measure and evaluate. Consequently, when \ce speak of 
measuring such human qualities as problem solving in mathematics, nr 
teacher effectiveness, or vocational aspirations, ve are speaking of 
a process that is vastly different from that of measuring electric 
power output, or the noise level :n communication lines, or the 
trajectory of a missile. Indeed, the difference is so great that an 
atomic scientist concerned with measuting the speed of electrons once 
suggested to me that v:e should probably drop the word measurement 
altogether when dealing with educational and psychological phenomena. 
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He may well have been right, for I suspe :t that much of the 
misinterpretation and over-interpretation of test-score data that 
bedevils so much educational thinking stems from the failure to 
realize that the metaphor of tne yardstick, or the chronometer, or 
the ammeter, or whatever, is a wholly inappropriate metaphor when one 
is trying to evaluate pupils 1 development and the educational programs 
and environmental conditions that affect it. 

i do not intend the foregoing to mean that, in some appro- 
priate sense, the measurement o f pupil performance is a hopeless or 
futile endeavor. Quito the contrary! Ftir t^ermor* , such measurement 
is indispensable if we ever expect to render a rational ralhei than 
a purely intuitive accounting of how schools and school systems are 
doing. But the r r ndei ing of such accounts in education is riot li ely 
to be very sound or instinctive if educational decision-makers think 
that assessing the quality of human learning and development is on 
all fours with measuring the quality of widgets. 

IV 

To recapitulate briefly at this point, what sort of perspec- 
tive on the evaluat ' jn/accountability equation do tiie three bits of 
history provide? First, the Puritan Fathers who wrote the lid Deluder 
Law were so sure of their educational objectives and the means by 
which they were to be attained that they were able to get *wav with 
holding every school district accountable for providing a particular 
type of ins truct ior al service, They did not, however, cotizern them- 
selves with the evaluation of the effects of the instructional service 
provided, since th^y assumed that that would be taken care of by more 
remote means on the Day of the Last Judgment. They were apparently 
unaware of the possibilities of evaluation as a form of s elf -correcting 
feedback. 

Back in 1930, 1 was held accountable for producing a certain 
single measurable result, and by that result my perf orman :e was 
evaluated. There was, however, no obligation upon me to iccount for 
the means by which 1 obtained the result. The feedback was sure and 
swift, but it was vhat Norbert Wiener would have called defective 
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feedback inasmuch as it included no information on any side-effects or 
after-effects my teaching methods may have been having on the student. ^ 

ESEA holds school districts accountable for rendering an 
accounting — that is, for providing an evaluation -- of the effects 
of the programs being Federally funded, but it says nothing about any 
punitive action that might be taken it the hoped-for results of the 
programs are not forthcoming. That is, it calls for effective evalua- 
tive feedback — which incidentally it has not yet been able to get in 
any comprehensive way — but it docs not specify how the feedback 
would be used if it were obtainable. 

In looking back over these three aspects of the role of 
evaluation in the eva luation/acccunt abi lity equation, one gets the 
feeling that something is missing and that that something is to be 
supplied by a reversal of roles. In addition to thinking of the vole 
of: evaluation in an accountability system, one needs to think also 
of the role of accountability in an evaluation system. Vhich Is to 
say that if educational evaluation programs are to serve any useful 
educational purpose, then those who support and manage school systems 
must be made accountable in three ways: (1) for seeing to it. that 

the evaluative information the programs provide is as good as it can 
be, (2) for seeing to it that the information is interpreted within 
the limits imposed by the nature of the data, and (3) for seeirg to 
it that the information is used in some systematic fashion to find ways 
of cent inually bettering the quality of instruction for all the chil- 
dren in all the schools. 

A final comment or two on each of these three points is now 

in order. 

1 . How to make su re that the information an eval ua tion 
program prov i des is a s good as it can be . This means first of all 
selecting tests and other instruments that are well-crafted and well- 
validated for the purposes to which they are to be put. There is a 
considerable; body of literature on how to make such selections and an 

g 

even larger body of measures from which to select. This material 
should be conscientiously examined before picking any test for use in 
the schools. Second, it means that the tests shall be administered in 
a manner that guarantees, insofar as possible, that the students know 
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what they are expected to do and that they will do the best they can. 
This may seem painfully obvious, but the fact of the matter is that 
test data is too often invalidated right at the source because of 
maladministration. Finally, and equally obviously, the tests must be 
scored with scrupulous accuracy. I mention these humdrum rules only 
because I am impressed by the fact that the failure to observe them 
is usually overlooked as a possible explanation of why the pupils in 
some schools appear to perform surprisingly higher or lower than their 
counterparts in othei schools. 

2 . How to ma ke su re that the results a rc inte rpreted wi thin 
the limits imposed by t h e nature of the data . Here ve are in con- 
siderably deeper trouble: because it is abundantly clear that most 
consumers of achievement test results seem to be amazingly unaware 
of the limitations of such data. One of the glaring problems in this 
connection is that of getting those who make educational decisions on 
the basis of test scores to realize that the best of achievement 
tests is never more than a sample of a student’s performance and is 
therefore inevitably subject to sampling error. This simply means 
that if nis score on, say, an arithmetic: test places him among the 
bottom third of his classmates today, his score tomorrow on an alter- 
nate form of the same arithmetic test has a good chance of placing 

c 

him among the middle third of his classmates." Failure to recognize 
this inherent bounciness of test scores can and does lead to all sorts 
of mistaken conclusions about the effectiveness of remedial programs 
for students who are selected for such programs on the basis of their 
low achievement test scores. 

Another glaring problem in the interpretation of academic 
achievement tests has to do with the kinds of numbers in which the 
measures are customarily expressed — namely, so-called grade equi- 
valency scores. Except for the notorious IQ, these are probably the 
most convenient devices ever invented to lead people into misinter- 
pretations of students’ test results. Both the IQ and grade equi- 
valency scores are psychological and statistical monstrosities. I 
have defined the IQ as "a dubious normative score wrapped up in a 
ratio that is based upon an impossible assumption about the equivalence 
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of human experience and the opportunity to learn," A grade equi- 
valency scoie has many of the same properties, and as such it lures 
educational practitioners to succumb to vhat Alfred North Whitehead 
called "the fallacy of misplaced concreteness 

There is not enough time here to go into all the irration- 
alities that underlie the construction of grade equivalency scales, 
nor all the misconceptions they generate irt the public mind about 
vhat achievement tests are saying about how well students and schools 
are doing. Instead, I urge you to read a recent brilliant paper by 

1 2 

Roger Lennon, entitled "Accountability and Performance Contracting." 
Lennon’s credentials are among the best, since he is senior vice 
president of the company that publishes two of the most wide ly-used 
achievement test batteries — the Stanford and ;he Metropolitan -- 
both of which are well-fitted out with grade equivalency scales. I 
have said the paper is brilliant; one might also call it courageous, 
because in it, Lennon, from his own intimate knowledge of the subject, 
spells out in grim detail just about everything that is absurd, wrong, 
and misleading about grade equivalency scales and why they should not 
be used in assessing professional accountability or in determining 
how much educational contractors should be paid. 

In his frank discussion of this and other similar problems 
in the interpretation of educational measurements, Lennon nicely 
exemplifies an important aspect of the role of accountability in 
educational evaluation. 

3 . Finn l ly , how t o use evaluative data in a sys te matic 
fashion to find ways of c ont inually bettering instruction for all the 
children in all the s chools . This, it seems to me, is the major task 
that lies ahead, if educational evaluation is to fulfill its promise. 
And it brings me to the questions in your conference program that 1 
am expected to answer. I shall now answer them: 

1. Can the relevant inputs, outputs, and conditions of 
operation [of educational systems) be sat isfact or ily measured? The 
answer is, "Yes, for the most part they can be, if school systems will 
make the kind of informed and serious effort required." 
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2. If so, what are the appropriate techniques? And the 
answer is, "Read my extended remarks on this subject in the Phi Del ta 

13 

Kappan of last December," 

3. If not, what remains tc be done? I have already said 

that adequate evaluative techniques are available if one has the will 
to use them. Nevertheless, it must be said that we do need better 
measures than we now have of the personal-social development of stu- 
dents, better measures than we now have of the many factors inside 
and outside the school that influence students’ overall development, 
and more particularly better ways of observing and describing what 
actually goes on day by day In the teaching-learning process. By 
this I mean that we need far better ways of systematically monitoring 
and describing what is really going on behind the facade of fancy 
labels by which we characterize so many so-called innovative programs 
like I.T.A., G.S.A>, M.B.O., the Open Classroom, the Discovery 

Method, and so on ad infinitum. I am convinced that ve ca n obtain 
these kinds of information if we have the will to do so. 

4. finally, are different techniques needed for different 
types of educational systems? And here my answer is, "Yes, Out .... 11 
Yes, the evaluative techniques one would use for a small homogeneous 
educational system would be different but also less satisfactory 
than those one would use for a large heterogeneous system. But the 
best way for small homogeneous systems to secure the most useful 
evaluative data about the effectiveness of their educational programs 
is to join forcer., for evaluative purposes, with other systems, pos- 
sibly on a state or regional basis, so as to enhance the possibility of 
uncovering, through w r e 11-worked-out statistical analyses involving 

all the schools, those educational innovations that have the best chance 
of paying off for their own students. 

The last answer is meant to imply that an evaluation system 
expressly designed to keep the quality of instruction continually 
rising will be a highly complex system. One might prefer something 
simpler. But I suggest that, in the highly complex world in which we 
now have to live, simplistic approaches are not likely to help us much 
in finding our way to education for either the good life or the good 
society . 
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T H K Y U T U R K OF A C C 0 U M T A H I J, I. T Y 



John W. Porter 



There are three aspects to the topic I am to discuss at 
this afternoon session. But before talking about these three 
aspects, a general definition of accountability in public education 
seems in order: 

Accountability is not performance contracting. Account- 
ability is not progiam budgeting (P.P.B.S.). Accountability is not 
cost effectiveness. It is not testing nor is it merit pay for 
teachers, or e mean;: of relieving teachers of their jobs. 

Accountability ia the guarantee that all students without 
respect to race, income, or social el?ss will acquire the minimum 
school skills necessary to take full advantage of the choices that 
accrue upon successful completion of public schooling, or we in 
education will describe the reasons why. 

What accountability probably means to the adult layman is 
returning in part to what existed in the 30 Vs and '♦ 0 1 s ; a move away 
from the permissive days of the bO’s and 60*s. But this time instead 
of the ’’Produce, Slide Through or Fail” responsibility being on the 
student, the accountab i 1 i t y emphasis envisioned a:> a "produce or 
change" concept assigned as the responsibility of the educational 
O estab lishment . 
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For a moment, let me share with you the; beliefs that I 
have, and that I believe we should all have, in regard to public 
education, and why there is a need for educational accountability. 

First, I believe that public education must guarantee 
that nearly all of the young people -- those children in our elemen- 
tary schools — will acquire competencies in the basic skills of read- 
ing, writing, and arithmetic, regardless of their socio-economic back- 
ground. This does not mean any leveling off on the development of the 
whole child. It does mean altering the educational delivery system 
in whatever way is necessary to ensure that the daughter of tne 
unskilled ghetto worker gains from the kindergarten the educational 
choices that presently accrue to the son of a college professor. 

Second, I believe that our public education, particularly 
in the secondary schools must be programmed in such a way that the 
students will feci their secondary school experience is equipping 
them to be effective citizens in the adult society of the 21st 
century. We should be concerned when we see that perhaps two- thirds 
of all the work we do in our secondary schools is done to prepare 
35 percent of our young people to go to college, when at the same 
time, nationally, we have a third of our entering ninth graders fail- 
ing to graduate. 

For counseling effectiveness, we need to strongly consider 
the use of public relations persons on loan from business and industry 
to the secondary schools to supplement the professionally-oriented 
counselors. If the status of the world of work is to change to meet 
existing manpower needs, and if we are to demonstrate that everyone 
doesn't need to go to college to teach, wc could well benefit from 
this "outside” contact for cur pupils on a regular basis, not just the 
"career day" type of exposure. 

We should also be concerned about the accountability of a 
system that seems to get the 6*5” basketball or football star through 
the academic mazes and to an attractive salary, while being ill- 
equipped to meet the needs of his 5'6” brother. 

Third, I believe acceptable public education is going to 
require that we educators be responsible for seeking out, estnblish- 

o 
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ing and coordinating effective programs of adult continuing education 
which meet the needs of welfare mothers, the underemployed, the 
housewives and the everyday workers that want some vocational skills. 

When our educational system is so streamlined and so 
exceptional that it is able to respond to the needs of most cf our 
200 million citizens in regard to these goals, then and only then 
will we be carrying out our educational committment to the citizens 
of our country and be achieving a degree o f accountability. 

Dr. Lessinger, former Associate Commissioner for Elemen- 
tary and Secondary Education in the U.S. Office of Education, stated: 

'Today the questions asked focus on results obtained for 
the amount of resources used, and whether or not taxpayers 
are in fact getting their money's worth. The quest ions 
are pointed, insistent and abrasive. I for one welcome 
the questions and hopefully wo as a profession will want 
to respond to them with alacrity. 

The challenge is clear in my mind and I hope in yours. 

Ke must start to guarantee student performance, one aspect of 
accountability in tne future; and you don't do this; by instituting 
remedial programs to correct deficiencies in secondary schools. We 
must begin to guarantee year by year growth, starting in the elemen- 
tary schools. Such an undertaking presupposes clearly spelled out 
performance objectives and criteria references for measurement. 
Criterion references for measuring student performance would pre- 
suppose an agreed-upon level of competency in tasks that were being 
undertaken by the students. 

Many of the principles underlying performance contracts 
and the more general concept of accountability when put together 
are worthy of consideration and utilization by all teachers. We 
will have accountability in the future. Accountability should be 
welcomed by the teaching profession, since the ultimate result is 
improved teacher performance and possible Increased teacher salaries, 
not abdication o' professional prerogatives. 
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Several aspects of accountability we can expect in ihe 
future which are currently being looked upon with skepticisn are: 

1. Paying for results rather than promises. 

2. Lesigning performance objectives to evaluate the 
instructional procedures. 

3. Identifying each student's characteristics and 
entrance level. 

4. Specifying in advance desired outcomes of individual 
student performance. 

5. Testing the instructional sequences to sec if they 
achieve what they purport to achieve. 

6. Reordering instructional strategies and personnel 
based upon student needs, abilities, interests, and 
attitudes . 

7. Involving the parents of the community in the educa- 
tional process right in the classroom. 

0. Informing students, parents > and taxpaying citizens 
what we can and cannot do in a given situation and 

t 

why. 

These eight factors are difficult to refute. They answer! 
the very basic question of "What if a student does not reach the 
ob Jectives?” That is, we as educators have to be prepared in the 
future to tell students, and theii parents, that the student hasn’t 
achieved; he needs more summer work, or extended day or week help,: 
or the diploma he will receive is for attendance, not achievement. 

r 

Accountability of the future means not passing students from level' 
to level because of chronological age and presence* in the daily 
classroom. ’ 

The eight factors cited are difficult to incorporate in ;o 

( 

everyday classroom use given the way classrooms are now organized, 

final analysis is nothing more than tetter 
in the classroom, by the principal in I is 



But accountability in the 
q inagement by the teacho 
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or her office, and by the superintendent at his conference table. 

For this simple reason, accountability will become almost a house- 
hold word and acceptance is the future of accountability that is 
assured . 

In the December 1970 issue of the Phi Delta Kapp an , Myron 
Lieberman, as guest editor, wrote: 

"If the public schools do not develop acceptable criteria 
and procedures for accountability, they will stimulate the 
emergence of accountability through alternative school 
systems, i.e., the voucher system. To put it bluntly, if 
school systems do not begin to do d better job cf relating 
school costs to educational outcomes, they are likely to 
be faced with a growing demand for alternatives to public 
schools. These alternatives may not be better — and may 
be even worse than the public schools. Nevertheless, it 
is difficult to see how public school educators could 
argue this point effectively unless and until they develop 
more effective ways of being accountable to their patrons." 

Accountability , whether or not we want it, is going to be 
a part of the educational scene in the 70's. The important issue 
for teachers and administrators is that the failures of the past 
and present cannot be allowed to rest solely upon the shoulders of 
the educational community. If we accept this, then let us look ai 
these three questions: 

I. What educational improvements 15 , it reasonable to 
expect fer the future application of techniques of accountability 
and how will they be obtained? 

II. What arc the probable sources of resistance to account- 
ability, and how can such resistance from wit .in and f t om outside the 
educational . ns ti tut ion he overcome? 

HI. What important defects ? 1 the educational system ei’e 
likely to ref ain unaffected by uc conn tab i ' i t v 7 



o 
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Let us now review some of the possible educational improve- 
ments which migh 4 ; come about as a result of using techniques of 
accountabi 1 i ty . 



I . 

Schools traditionally have not been problem-solving agencies. 
Schools traditionally have not focused upon cost effective management 
techniques in the classrooms. And most schools have net been held 
responsible for student performance. 

Future improvements in education as a result of innovative 
techniques will be based in part upon the development of two specific 
types of information by local school districts: 

1. Improved and more comprehensive student performance 
measures in the cognitive as well as affective domains. 

2. Improved and more specific performance objectives 
related to the functions and contributions of teachers, 
principals, administrators, school boards, and the 
parents of students. 

At present, such information does not to a £reat extent 
exist in school systems. As a result, a major consideration in 
moving toward accountability must be development of d<*;ta gathering 
information systems and analytical assessment of the data gathered. 

If properly managed, such an arrangement should result in 
a school system operation ba^ed upon some clearly spelled out 
objectives. Felix M, Lopez labeled this "Management ay Objectives" 
in a recent article entitled "Accountability in iducetion." 

This process requires a school district: 

1. to ideutif> the common goals at all g'ade levels for 
all subjects provided: 

2. to think through its management procedures or delivery 
system in terms of pre-testing and post-testing as 
they relate to responsibilities of teachers; 
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3. to evaluate each student's performance in accordance 
with some overall efforts, or specify why such per- 
formance cannot be achieved. If we fail to evaluate, 
while v T e may know exactly what we are doing, we will 
never know what we ha ,r e done; 

4. to assure that school district goals are translated 
into performance objectives understood by students 
and parents alike; 

5. to reach an understanding of steps to take when the 
child does not reach the minimum level of proficiency 
at the originally agreed-upon specified lime. 

To amplify or clarify these points in terms of educational 



improvements which might be derived by the application of techniques 
of accountability, one needs to look at what our common goals are in 
terms of ''grade level 1 ' performance. In essence, four educational 
improvements should emerge: 



1. Improved teacher classroom management and professional 
performance. 

2. Improved student academic achievement, especially by 
the lower half of the classroom distribution. 

3. Improved student attitudes and behavior. 

4. Improved reporting of student progress in ter 'is ui 
s tuden t~sckool-~co.:;mun i tv relat ions . 

further techniques of accountability should help remove the 



"blackboard curtain" created by the construction of classrooms on a 
30 to 1 basis. Accountability to be effective will have to permeate 
through the closed-door class* oom. Thus, each teacher working with 
parents and others at each level will have to decide what exactly 
are the classroom expectations. In the fourth grade, for example, we 
must ask, T, What is it we want linn th graders to know when they have 
finished a year in our classroom'" 



o 





This concept of accountability focuses upon educational im- 
provements by level and subject, and as some have suggested could 
result in a marriage between technology, and personal pedagogy, with 
the emphasis on measuring individual student progress. 

Another dimension of the future of accountability for 
improving education must result in less student absenteeism, fewer 
dropouts, less special education, less fear of actually failing a 
grade, or le : fear of ''sliding through" feeling inadequate for 
the next level, lower teacher turnover, and less family mobility 
during the school year. 

The improvements I have describ 1 will ba obtained through 
Iol il initiative resulting in a reor*’ ring of priorities, from 
successful performance contract arrangements, from new leadership 
directives, from state departments of education, and from state and 
federal appropriation specifications. 

Let us now talk about our second basic question — who 
will oppose accountability? 



II. 

There are significant numbers of individuals in at least 
eight groups that may oppose the concept of accountabi li ty as I 
have defined it: (1) students, (2) local school educators, (3) 

central administrative staff, (4) school board members, (5) tax- 
payers, (6) legislators, (7) teacher training instructors, and (8) 
state department of education personnel. 

Some students may resist the concept since it will focus 
on their performance in certain areas. Common educational objec- 
tives are desired; however, when these conflict with individual 
student preferences, an accommodation rust be reached. Such 
accommodation however does not mean acquiesce ig, but spelling out 
in clear, precise ljnguugL the alternatives available. 

Seme teachi.s r uul support the accor nlabi li ty concept 
because it implies ilia* their r *ork i , ng t^ahuiled -- and this 
is disconcerting to some individuals. 
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associations may oppose the concept on the basis that it implies 
an evaluation of the entire teaching profession. 

Some central administrators, including middle management, 
may resist the concept of accountability — not because of a desire 
to avoid involvement, but because it may imply that outside assis- 
tance be brought in. This assistance may be a threat to the 
established practices of administrators. However, one of the major 
fallacies of educational management is that all, or nearly all, 
schools must be run in the same manner: rhey start at promptly 

8:30 a.m. and close at exactly 3:30 p.m.; students are enclosed in 
units called 'classrooms" except when they are allowed outside for 
recess or to pass betvreen classes; all students are given the same 
curricula; and so on. The accountabili ty concept may seriously 
challenge standardized practices — particularly in school systems 
when significant proportions of students have been shown to be 
fai ling . 

It is likely that s-hocl board members will generally 
favor the accountability concept as it holds the promise of 
alleviating educational problems at little cost; however, if the 
concept is seen as one that requires additional monies, it is likely 
that many school boards will balk at the idea. Local taxpayers, too, 
will favor the idea — so long as it does not cost additional tax 
dollars . 

State legislators are a mixed lot of ideologies and 
experiences, and they carry a variety of expectations for the 
schools. It is difficult to predict their feelings as a group -- 
however, they will carefully scrutinize any concept that may cost 
additional monies and one sense:; that they are currently not as 
appreciative of how well the public schools are working as they 
might be; in some situations with justification. 

Teacher training institutions are frequently wary of 
innovations . It seems as if evaluations are conducted, but we too 
seldom sec actual changes in practice. Why does this occur? 

Who, or whit, stills the program? It is likely that increased 
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accountability in the elementary and secondary school settings 
will result in increased pressure on the teacher trainers and 
their administrators to turn out more graduates who can guarantee 
performance. 

Finally, some staff members of state departments of educa- 
tion will resist the concept because it will mean a drastic reorder- 
ing of priorities and activities for them. The states are thought 
by many to be constitutionally responsible for education. If states 
are to take a leadership role in exercising this responsibility, it 
is likely that at least six implications will emerge: 

1. State departments may be required to standardize 
educational assessment of pupil progress. 

2. State departments may be required to develop uniform 
local budgetary procedures. 

3. State departments may be required to establish 
procedures for equalizing financial resources by 
district . 

4. State dep«,rtments may be required to adopt guidelines 
for the reorganization of school districts. 

5. State departments may be required to get involved in 
teacher negotiations. 

6. State departments may be required to move from locally 
defined regulatory service and consultative subservient 
agencies* to monitoring and management support agencies. 

Chief State School Officers will have to assume a leadership 
role not only in establishing in-service training for their own staffs, 
but also for encouraging regional staffs within their states to tune 
in, as well as establish immediate discussions with the various 
professional groups d:\rectly affected by the concept. 

In responding to the second part of this question, "et me 
state, there is no panacea to overcome the resistance to accountability, 
however, the complete involvement of those directly affected will help. 
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Aaron Wildavsky, writing in the Phi Delt a Kappa n journal in December 
1970, is right when he states, "No plan for accountability can succeed 
unless all the major participants in the educational process .. .see 
something in it for themselves." 

Many good teachers may, with the proper involvement in 
accountability, overcome the emotional trauma of having a class of fail- 
ing students, if shown how such techniques can provide direction and 
support against arbitrary administrative decisions. At the same time, 
principals may begin to view accountability as an added leverage for 
dealing with the ineffective teacher. The other six groups of the eight, 
once involved ana when we have identified clearly the specific benefits 
for them, may accept the pain of raising more money, for example, rather 
than opposing the concept. Different strategies and forces would be 
the deciding factors, based on the local and state conditions. In any 
event, communication in regard to the accountability concepts must be 
conveyed in such a way that all groups can accept the ultimate 
objectives, improved educational performance, at a cost which can be 
j us tiffed . 

Ve have talked about what accountability can do, and how to 
go about getting it, and we have talked about some of the difficulties 
of obtaining accountability. 

Let’s now look at what are some of our problems that 
accountability cannot overcome. 



III. 

As mentioned, implementation of the accountability concept 
v 111 not alleviate all of the problems of our educational system. A 
number of vexing socio ducational views will remain, including; 

1. the issue of how monies should be allocated to schools 
in order to best facilitate equality of educational 
opportunity ; 

2. the issue of how educational monies should be collect- 
ed in order to best facilitate an adequate and fair 
source of school support; 
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3. the issue of how teachers should be certified to 
teach in the schools in order to facilitate our best, 
students going into the professions with the best 
possible preparation; 

4. the issue of cons truct itig school facilities that will 
adequately and fairly serve the next generation of 
students ; 

5. the issue of how the often ponderous educational 
bureaucracy can best be organized so as to facilitate 
a new sense of urgency and of innovative leadership 
that will respond more adequately and quickly to 
societal needs; and 

6. the complex issues surrounding student disinLerest and 
disaffection which mirror a more pervasive societal 

ci is is. 

In summary, I have defined accountability of the future as 
a quality or state of education whereby educational institutions take 
r es pons ib i li ty for ensuring that their students reach agreed-upon and 
clearly-defined educational objectives. I have further discussed two 
aspects of accountability: (1) possible benefits to the educational 

system that may result from widespread adoption of the concept, and 
(2) possible sources of resistance to accountability. In addition, I 
have spoken briefly of the problems that face us — and will still 
face us even if we attempt to hold our schools ,r aceou?n able . ° 

Let me conclude by stating that I think the movement toward 
accountability in education can be a healthy one, as it can help to 
ensure that a_H children will be served hv the schools. However, let 
me also close with a warning: accountability is not a panacea; the 

major problems of this society and its schools will not be solved 
without a national, state, and local reordering of priorities ant' 
without an equalization of the educational, social, and political 
oppor tuni t i es available to our children, youth, and adults. 

o 
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Maybe the most beneficial outcome of the future in account- 
ability will be a i complete shift in the role of the school, which has 
up to now professed to be committed to meeting the needs of all of 
the children of all of the people. This possible overstatement, sad- 
ly to say, is one of the big reasons for the current controversy over 
public schools. Accountability, more than any other single concept, 
will in the future force all of us as educators to examine this all 
embracing goal or American ideal. We need to ask ourselves, "Are 
there institutions other than t lie school that might be or could be 
used to assist some of the children of some of the people in accom- 
plishing certain tasks? 11 

The future of accountability, whether the emphasis remains 
on efforts to relate ,T educat ional inputs 1 ' to "student output, M or 
whether the emphasis is on patron choice, that is vouchers, free 
schools, open enrollments or parochiaid, school officials will in the 
future have to face each issue by answering clearly to six specific 
ques tions . 

What are the common and spe c i f 1 c goals to which the 
teacher and school are striving? 

Wha l s tudent , co mmu nity o_r s oc i e ta_l needs inventories 
are available, on paper, to indicate change strategies 
which should be unde rt alien? 

Wh a t spccjf ic anu measurable p er f o r man c £ objectives 
have been written down that would enable parents, 
students, and teachers to understand the minimum 
expectations of the unstructured progress? 

What analysis of the existing del i ver y system is avail- 
able to indicate that the current educat ional input 
approach is manageable and defensible is compared to 
the alternatives? 

What f o r n s of t os t ij iy; and e y a I u a it i_c > 1 1 will be undertaken 
to enable the n at large community" to know whether or not 
the delivery system measured up to the performance 
predi ct ions? 

o 

O 
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6. What recommendat ions are the school systems ready to 
make as a result of the testing and evaluation data? 

Perhaps we have always had accountability -- we always 
checked out what went into education — facilities, materials, warm 
bodies, hot lunches -- but seldom did we worry about what came forth; 
what pupils learned; what skills were obtained. In fact, we went out 
of our way to find excuses for those children who did not learn 
broken homes, language barriers, ethnic or national background, 
malnutrition. That is, we placed toe much responsibility for success 
upon the student and his parents. But, if the student didn't perform, 
we began passing him up the educational ladder anyway. What is 
envisioned now is a strengthening of the role of the teacher, so that 
he or she is not placed in such a situation. The future, as account- 
ability becomes firmly entrenched, wi.ll allow for very few excuses. 

We educators will be responsible for failure, and the exciting, 
fantastic goal before us is to have achievem ent realized by nearly the 
total school population. 
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PUBLIC EXPECTATIONS 
III 



Scarvia B. Anderson 



ACCOUNTABILITY: LA NOUVELLE VAGUE?* 

This spring the National School Boards Association is holding 
five regional conferences on accountability; the American Management 
Association is holding three; Educational Testing Service is having one 
in Washington and one in Los Angeles* There have been countless other 
sessions sponsored by such diverse groups as the National Committee for 
Support of Public Schools, Harcourt , Brace, and Jov anovich, the Center 
for Urban Education, and the Ohio Division of Guidance and Testing. 

Journal editors — recently for the Phi D elta K appan and 
Educat ional Te chnol ogy — are devoting whole issues to accountability. 

Foundations and federal agencies are allocating hundreds of 
thousands of dollars to feasibility studies of voucher systems, 
experiments in performance contracting, and that new area of scientific 
and philosophical inquiry, meta-accountability. Cft was inevitable 
with all of the accountability talk that there would emerge a willing 
and transcending profession to talk aboujt^ the accountability talk.) 

Only a gasp before it announced its need to cut back some 40 
million dollars 1 worth of services foe the rest of this school year, 
the New York Lit> Board of Education, in cooperation with the United 

^Speech presented at the Hollywood, California, conference in the 
unavoidable absence of l)r. U. T. dames. 
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Federation of Teachers, told the press that it was planning to let a 
hundred thousand dollar contract for "an accountability design,” 

khat is this tiling that is causing educators to do so much 
traveling and eat so much rubbery chicken with cold mashed potatoes? 
That is pushing preschool education, computer assisted instruction, 
sensitivity training, black studies programs, and :he National Insti- 
tute of Education off the pages of the journals ani the newsletters? 
That is readily prying funds from close-fisted agencies? And that can 
bring even one school board into close harmony win a teachers union? 

Let us hear first the words of Saint been Lessing*. r, of 
course, who, if not the father, is certainly the preset of the new 
cult : 

[Accountability is] the process desiguo< to ensure that nv 
individual can determine for himself if the schools are 
producing the results promised (1970, p. 52). 

On a latei occasion, Dr. Lessinger invoked his Principle of Public 
Stewardship through Accountability: 

Independent, continuous and publicly n ported outside iwie.% 
of promised results of a bureaucracy promotes compel en< i and 
responsiveness in that bureaucracy (1971, p. 11). 

Although Lessinger started his definition oi "accountability" 
at a level of complexity and application considerably beyond the s i i:,y i e 
statemenl in Webster’s Collegiate -- "to be accountable" is to be 
’’answerable" or "explicable" -- the explication of the concept has i * 
come a major professional occupation, 

Lieberman notes that, in spite of variations in def’uiions 
of M accc unt abi lity 

At a common sense level, there is ticc wilt abi I i ty when re scare 
and efforts are related to results it wn.es that are useful 
for policy making, resource allocation, or compensnt ion (39/0 
p, 194). 

!5arro does not question the "general meaning end tr-poii ior 
tiie sell jo Is": 

o 
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...schools and sdiool systems, or more precisely, the 
professional educators who operate them, should be held 
responsible for educational outcomes — for what children 
1 earn .higher quality education can be obtained by making 
the professionals responsible for their product ... (1969 , 
p. 196). 

President Nixon made accountability "official 1 ' in his 1970 



School administrators and school teachers alike are respon- 
sible for their performance, and it is in their interest as 
well as in the interests of their pupils that they b? held 
accountable. 

The emptiness of such a statement is striking without the 



specification of exactly who is responsible for wha t . Henry Dyer, of 
all the expiicators, has dealt most thoughtfully with his problem. 
His definite f "accountability 11 embraces three general principles: 



1. The professional staff cf a school is to be held 
collectively responsible for knowing as much as it can 
(a) about the intellectual and personal-social develop- 
ment of the pupils in its charge, and (b) about the 
conditions and educational services that may be 
facilitating or impeding the pupils T achievement. 

2. i 1 ic professional staff of a school is to be held 
collectively responsible for usjtig this knowledge as 
best it can to maximize the development of its pupils 
toward certain clearly defined and agreed-upon pupil 
performance obj ec t i ves . 

3. The board of education has a cort espond ing responsibil- 
ity to provide the means and technical assistance where- 
by the staff of each school can acquire, interpret, and 
use the information necessary for carrying out the two 
foregoing functions (IV 7* , p . 206). 



Educational Message: 
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In addition to emphasizing process rather than product, Dyer restricts 
himself to the school as the unit of observation and labels short-term 
efforts to demonstrate accountability as exploratory at best. The only 
difficulty in Dyer’s argument lies in the definition of "a school.” 

Hartnett (1971), in a paper soon to be published, will help 
clarify matters further for observers of the newly systematic educational 
scene by spelling out the differences between educational accountability 
and educational evaluation. Many educators felt that the latter was 
still not secure in their vocabularies -- much less their practices — 
when the "new wave” hit. Fortunately, they do not have to be bowled 
over completely. Accountability and evaluation are both concerned 
with the effects of educational programs with whether they are meet- 
ing their objectives. They both utilize measures of educational input 
arid output and documentation of the "treatment" and surrounding 
conditions. They differ in two main ways: 

1. Evaluation is concerned primarily with effectiveness 
(the degree to which the institution or system succeeds in doing what- 
ever it is trying to do); accountability is concerned with effectiveness 
and e f f iciency (the capacity to achieve results with a given expenditure 
of resources). Thus the latter is even more complex that the former , 
since it must encompass nol; only alter?:;'? ; determine success but also 
how much it cost to obtain it and the relationship between cost and 
benefit . 

2. Educational evaluation -- though sometimes mandated . n 
general terms by a funding agency -- is largely the business and province 
of the educational institution or system, itself; and it stands to 
succeed to the extent that it is viewed by administrators end staff as 

a vehicle for program improvement. Aeeoumubi 1 i ty , o ». the other hand, 
carries with It the notion of external judgment and control. Ihe 
advocates of accountability view this js a positive feature — the tax- 
payers have a right to know, hut quoting Median, quoting one classroom 
teacher : 

If vv sa" that someone is accountable we usually mean that 
"he must suffer the consequences of his actions." ife hardly 
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ever mean the more positive "he will profit from the 
consequences of his actions" (19 70 , p. 13). 

Of course, as Roger Lennon has reminded as, the idea of 
accountability is not new; 

...at the University of Bologna in the 15th century, student- 
enacted statutes required that the "professor start his 
lectures at the beginning of the 'cook, cover each section 
sequentially, and complete the book by the end of the term"; 
if the professor failed to achieve the schedule, he forfeited 
part of funds that he hinse.lf had had to deposit at the 
beginning of the? term! (1971, p. 3). 

And a recent letter writer to the Phi Delta Kappan has noted that; 

The Education Code of Sierra Leone in 187b provided for a 
"result" grant of sixpence for each pass in an examination 
in the three R's. This policy was followed in Gambia, the 
Gold Coast, and Nigeria. The policy was an imitation of 
the English system which was abandoned in England in 1897 
(Sherman, 1970, p. 253). 



Not now either is the concern of the public with the quality 
of children’s education; In 1830, a group of Philadelphia workingmen 
surveyed the curriculum and found it wanting. Tliey said, it "extends, 
in no case, further than a tolerable proficiency in reading, writing, 
and arithmetic, and sometimes to a slight acquaintance with geography... 
(Cremin, 1951, p, 35). 

Ihe most unfair impression that accountability advocates 
might leave with those they are trying to proselytize is that great 
number. 6 of teachers and educational administrators have not felt -- do 
not feci — a strong sense of compassion for their students and 
responsibility for their intellectual development. But the social 
milieu and the educational problems of the l^ZO’s are so complex that 
it is no wonder that th<y are reaching for almost any rope that offers 
to sa\u then from their sea of i'rus l , ,.t i on . V.lutber a :count nb i li t y is 
their best hope for salvation, whether in banging on t j it they will 
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be swept further out to sea, or whether the energy they expend in 
grasping it will leave little left for climbing into a sound lifeboat 
that is just over the horizon — these are serious problems to be 
contended with. For accountability must be taken seriously. 

It is already a powerful force in education for at least two 
reasons: First, it has managed in a relatively short time to accumulate 

the trappings of a discipline: Parts of accountability have been 

delineated, the delineation of the parts has been reinforced by names 
for them, there ar«* roles associated with the parts, and some techniques 
have been offered for carrying out the roles. Second, accountability 
is a large enough vessel to hold the concerns of many parties to the 
educational process; even if they are not all sympathetic, they are all 
involved . 



Let us look first at the parts and then at some of the parties. 
Accountability has at least five major divisions or manifesta- 
tions : 



1 . Per formance contrac ting — establishing with a contractor 
a level of payment based on the level of student performance delivered. 
The contractor is usually a commercial company, frequently wiLh 
educational curriculum products to offer. Standardized achievement tests 
provide the criteria of success. Recently Newswe ek magazine predicted 
that 1 7 U school districts would spend 50 million dollars on performance 
contracting this /ear (1970 , p. 58) . There are those who point out 

that performance contracting is associated more with training in the 
industrial sense than with education in the broad sense. 

2. i urn -keying -- the process whereb) a program established 
under a performa ice contract is adopted b> a school system and operated 
by its personnel. Some performance contracts specify the cost and 
effort required for turn-keying. 

3. Auditing -- the independent examination of an educational 
effort or perfo 'tnanee contract to verify rosulLs, check on processes 
personnel, and .regress, and -- frequently -- make an independent report 
to an interesto. external agency. More auditors than performance 
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tractors seem to come from nonprofit agencies. The demand for indepen- 
dent audit seems to be directly related to the iistance between the 
program and the funding source. 

4. Education vouchers — allowing education of children to 
be bought by parents in a “free market," through vouchers provided by 
school district officials or government agencies. This plan is 
associated primarily with James Coleman 1 economist Milton Friedman, and 
the Harvard Center for the Study of Public Policy. It implies, in 
various of its proposed forms, regulations relating to selection of 
students, access to the schools fur financial and program audit, 
standards of educational quality, and availability of evaluative data 
to potential purchaser-parents. The accountable party is the 
independently operated school. 

5. Incentive pa y — paying teachers on the basis of the 
performance of their pupils. This harks back to earlier century 
practices of the type already cited, and so fai it has met with little 
more popularity than any of the other merit-pay schemes advanced in 
recent years. Kenneth Clark of the Metropolitan Applied Research Center 
has made more headlines than headway in attempting to implement such 

an incentive pay plan in District of Columbia a choc Is . 

Coming to be more and more identified with “accountability" 
are another five activities on concepts. They come from other 
philosophical and operational sources with which they continue to be 
associated. They are; 

1 . Behavi ora l object i vos — s t a t en on t s of what the e d uc a- 
tional program is supposed to accomplish, the conditions under which 

it is to accomplish then, and the criteria whereby success in accomplish- 
ing them can be determined . 

2. P/f:S (Planning, Programming, Budgeting System) — a 
management tool first employed in national defense and designed to 
identify relationships between product outcomes and costs for various 
alternatives . 
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Needs as se ssment — a formal attempt: to determine the 
educational needs of a population or subpopulation. 

4. Sys tems analysis — actually a conglomerate of tech- 
niques associated with operations research and computer simulation, 
recognizing always the interrelationships of the components of a 
system. 

5. PERT (Program Evaluation Review Techniques) and other 
network-based management tools — tools designed primarily to assist 
administrators in monitoring the effective operation of an ongoing 
iys tern. 

The parties to the accountability push -- or debate — ■ are 
nore interesting than the techniques . They induce teachers, admin- 
istrators » minority groups, parents, psychometricians, and, of course, 
external observers: 

Teachers . Tessinger has predicted that in education’s 
accountable future the "teacher would become a manager, rather than 
a presenter of inf ormation" (1971, p. 57) . Fred Hechinger has 
explained the positive involvement of the United Federation of Teachers 
in implementing a plan to "establish procedures to nold the [New York 
City] schools and staffs accountable for their success in educating 
children" in terms of the lesser attractiveness of the alternatives: 

"Widespread difficulties in schools ... can create outright 

community anger which tends to arouse often irrational 

demands that the schools be held responsible for overcoming 







all. . . 


soc 


ia? ill 


s" (1971 


> P- 


7). 












le 


feels 


too tha 


V s 


y s tern-b 


ased att 


ernpts 


to 


upgra 


de per 


formancc 


are to 


DP 


pref v 


rred by 


the 


union 


to per fo 


rmant 


e o 


ant r ac 


t i ng w 


i th 


exte 


rna 1 


agencies 


or to the 


voucher 


system. 




















Robert 


Bh 


,ic r nan ♦ 


Di recto 


r of 


Res 


earch , 


At, ip r i 


can 


Fete 


ration 


of 


Teachers, sugges 


ts that 


accoun 


Labi L 


ity 


nay b 


e noth 


ing 


more 


than 


"pi 


i o in 


the eye" 


of 


Lcache 


ts. lie 


repo : 


ts - 


on a r 


esolut 


ion 


pas? 


ed by 


representatives 


of 


the Ted 


e rat i on 


in t. j 


rms 


of s u 


ch quo 


s t i < 


ons ci 


s these 



o 

ERIC 



yj 



K-9 



Can "the advocates 1 ' guarantee that performance contracting 
will not take the determination of education policy out of 
the hands of the public? 

Can they say, with a straight face, that performance con- 
tracting does not threaten to establish a new monopoly of 
education? 

How can they state that performance contracting would not 
subvert the collective bargaining process and reduce teacher 
input? 

Is performance contracting net predicated on the false 
assumption that educational achievement can be improved in 
the vacuum of a machine-oriented classroom, without changing 
the wider environment of the poverty-stricken child? (1971, 
p. 62). 

Deterline questions the "justification for expecting [teachers] 
to do better, or for holding them accountable for doing so... unless 
someone else accepts accountability for teaching those teachers 
relevant skills beyond those they already possess, and unless the 
conditions that limit their effectiveness can bo changed" (1971, p. 17). 

E ducational ad ministr ators. The Superintendent of Schools 
of Hartford, Connecticut, states unequivocally that state and local 
education governing bodies have no choice but "to take a leaf from 
bus iness . . .and refuse to develop and promote new educational programs 
and techniques, refuse to commit public funds, and refuse to employ 
personnel, until we first establish clear goals... , until we develop 
ways to measure accomplishment of these goals, and until we set up 
logical techniques to employ in reaching them" (1971, pp . 38-39). The 
Assistant Superintendent in Nashville, Tennessee, however, cautions 
that accountability for schools is different from accountability for 
other organizations (Ueck, 1971). In gener, l, school administrators 
seer more supportive of accountability than do spokesmen for other 
groups. Perhaps they agree that it is primarily an administrative 
innovation and sot an instructional one (Barrows, 1970). 
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B lack g roups . Kenneth Clark, although supporting some 
activities associated with accountability, has "warned that the 
accountability proposals would be seriously undermined if they are to 
be used 'as a semantic cover for the old alibis' of why lower-class 
children c nnot be expected to succeed" (Hechinger, 1971, p. 7). 
Represenfat ives of other black groups are stating that accountability 
is what they have been talking about, all along. But many add that 
they want it on their own terms -- and under their own control, 

Parents . This group, while increasingly vocal and active 
about the operation of schools, if one judges by newspaper and 
television accounts, does not appear to have much specific represen- 
tation on the panels currently arguing the case of accountability. 

Is It any longer safe to assume that, if their taxes aren't raised 
and if their children don't have to travel too far to school, aren't 
underfoot at unscheduled times because of school closings, seem to be 
learning something and staying out of trouble, and eventually get into 
colleges nr careers, they won't care what the magic formula is called? 

Psychome tr ici ans . Since test scores are viewed as the primary 
basis for determining whether educational objectives have been met 
and accountability established, it is only natural that those concerned 
with the properties of tests have had something to say abcut the 
matter. Mostly they have said that those letting and signing perfor- 
mance contracts are at best naive. Stake and Wardrop, for example, 
after reviewing t tie properties of gain scores, have concluded simply 
that "individual-student gain on a currently available standardized 
lest should not be used as a criterion of successful ins truct ion" 

(1971, p. 2). Lennon (1971) has pointed to the frequent lack of 
congruity between the behavioral objectives of a particular instruc- 
tional segment and tie kind of nationally norincd test that other 
stipulations of present performance contracts require, (Some have 
suggested substituting cri ter ton- referenced tests.) Other issues 
raised by this group include the validity problems associated with 
"teaching for the tests," comparability of alternate forms of tests, 
and the appropriate unit (individual, class, school, system) to which 
accountability procedures should be applied. 
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External observers . Fred Hechinger of The New York Times 
rejoices at least in the fact that the more sophisticated discussions 
of accountability recognize "that many factors contribute to a child’s 
record" and he alone cannot be held responsible for it (1971, p. 7). 
Sociologist Melvin Tumin said, in another context, 

It is sociologically axiomatic that when a number of parties 
are involved in any social enterprise, and when the enterprise? 
fails, each party will lay maximum blame for the failure on 
the others, and will assume only minimum blame, If any, for 
itself. As a corollary, it follows that the official verdict 
of gu.lt for failure will be imposed on that party who is 
weakest or least able to fend off the imposition of the 
official sti gma .... 

There are numerous ... evidences of the deep commitment of 
American education to blaming children for failing to learn 
as mu:h as the "standards" demand that they shall.... 

But all of this seems very much in the process of change... 
for nearly 20 years, starting just after World War II, the 
teachers of America, ar i their teachers, were attacked from 
ail e ides for the educational failures of children. Ihen, 
for a brief moment, until a temporarily successful counter- 
attack was launched, the families of chi ldren .. .were held 
to be essentially defective. 

Most recently, it is a combination of the educational estab- 
lish ?ient ... and of the corollary lack of community control of 
the schools that liar been made Lhe major scapegoat...- 

Whatever t ir supreme ignorant e on many key educational ques- 
tions may he , it seems quite clear ... that family life, 
community organization, aid the schools are all contributors 
to the educational outcomes of the children (19b9, pp. 7-9). 

It would be i aval i el* to conclude this overview of accountability 
without even n:ent ioning Texarkana, Arkansas, and Cary, Indiana. So they 
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will be mentioned — in the context of conclusions drawn frcm reviews of 
those performance contracts by another external observer, Minnie Perrin 
Betson. Mrs. Berson speaks regularly to thousands of teachers in the 
pages of the journal of the Association for Childhood Education Inter- 
national. She asks: "Is it really fair to expect Gary's schools to be 

flourishing oases In the midst of the many unresolved urban problems that 
surround them?... can outside education-mechanics bring in magical learn- 
ing solutions by converting a school into a skill-shop?" She continues: 



skills of reading comprehension and interpretation that 
differentiate mechanical mastery from fundamentals learning 
growth. I 

For the latter, more Is involved than taking over a school, 
bypassing teachers, hiring aides for one-sixth ofj the salary, 
and giving them fancy titles for checking the piecework in 
the child-learning-factory.... 

Educational accountability worthy of its name requires that 
teachers, administration and community be accountable to each 
other with honesty, compassion and determination (1971, p. 343) . 

Educational accountability has become a catch-alH for every- 



one's frustrations; many technical defects have been identified in 
applications of tin tools associated with it. Nevertheless, it is enjoy- 
ing a considerable v< guc*, and it is stimulating conversations between 
diverse groups concerned with American education. Where do vc go from 
here? Many possibilities exist. Three — for different reasons -- 
deserve special consideration: 



contracts with the kind of bad press Dorset t received fren Texarkana, 
the failure of capable organizations to devote their attention ?n 



Accountability is hardly achieved by simple test measures in 
which Mr. Lessinger so firmly believes. When children are 



continually given exercise sheets that resemble achievement 
test items, they can play the testing game with gveat savvy. 
So doing does riot assure that they have mastered -.ritical 



]. The first is the most cynical. A few more performance 
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refining present accountability tools and developing new ones, 
inadequate systems for disseminating information about appropriate 
techniques and training educators to use them, a degree of cumber- 
someness and expense associated with accountability ventures that 
makes administrators reluctant or unable to launch them, overemphasis 
on the engineering- f inancial aspects of accountability to the 
exclusion of the educational-personal ones, and predictions derived 
from the history of adoption of educational innovations — some or 
all of these could work to erase "educational accountability" from 
the vocabulary in a relatively short time, to be replaced perhaps 
by the name of a r.ew game for educators to play. 

2, Assuming that accountability is sustained by positive 
events, developments, and climate, then in a few years we might see 
a great many educational systems and institutions with more precisely 
defined objectives, indices and measures compatible with those 
objectives, systems for collecting and analyzing data longitudinal!;/, 
clear identification of who is accountable for what (with related 
schedules of reward and punishment), and efficient management system 
that facilitate operational planning and monitoring and associate cos 
with effectiveness. A rosy picture? It would certainly seem so. B i: 
let us pause for a moment to think about the fundamental emphasis of 
aceountabi 1 i tv . 

The fundamental .mphasis is on output . Many proponents of 
accountability would concern themselves with little else. Even the 
more sophisticated models that, mention input, only measure it as it 
exists, 'lhey do not raise the basic issues of the nature of the 
population to be educated, the present requirements of our highly 
urban- techno logical society, and the needs of the individual for 
personal fulfillment. In other words, proceeding from the basic 1 ir 
of thinking about accountability, the most brilliantly executed and 
successful der/onst rai ions of it stand little chance to do more than 
validate the present t due; ■ ional system -- to show that schools are 
doing a good job 'i what imy were supposed to be doing a long time 
ago . 
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In Education and the Cult o f Effic i enc y, Callahan talks 
about Bobbitt's approach to education: 

The standards and specifications for *- teel rails weri set 
by the railroads, not by the steel plants, and the specifi- 
cations for educational products should be set by the 
community, not by educators. A school system... can no 
more find standards of performance within itself than a 
steel plant can find the proper height or weight per yard 
for steel rails from the activities within the plant (1967, 
p. 83.) 



3. This leads to a third consideration about where we go 
from here. Is it possible that the current fire of concern about 
education that accountability lias helped to fan is at a sufficient 
height to lead to some receptivity to the idea of a drastic 
reformulation of education? Is it possible to invent a new system 
or series of systems 



o that is conceptualized and operated in the context of 
the demands society makes upon individuals and the 
opportunities it offers them, 

o that takes into account the characteristics of various 
populations to he educated and is committed to the 
development of individuals rather than to teaching 
certain subjects, 

o that recognizes that development encompasses a broad 
range of skills and talents, ranging from self- under- 
standing to i n t t rpe rsonal skills to advanced technolog- 
ical competencies (iirant , 1970) , and including the 
abilities to restructure society in the future, 



that is dedicat'd to tin* propositions that 
should continue t is ronghool a person's lifot 
edut alio: should not be (he responsibility 

soc ini i ns 1 \ t ! it 5 on , an J 



development 
i no and 
ot any single 
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o that is not constrained by structures and strictures 
from the earlier system, covered as it is by the Band 
Aids of lowering and raising compulsory school age, 
social promotion, and the many other attempts to doctor 
creeping irrelevance? 

Accountability leans very heavily on methods from engineer- 
ing, industrial management, and accounting. Wouldn't engineers, 
managers, and accountants prefer to lend their talents, along with 
those of educators, legislators, behavioral scientists, and other 
representatives of our society, to the enterprise of developing new 
educational models appropriate to the waning years of this century, 
rather than to dissipate them in the thankless task of pat chin., up 
or patinizing a system from another era? 
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